Bush spoke with PEOPLE about his new book of paintings, Out of Many, One, which features a selection of his portraits of American immigrants along with their stories

Days after George W. Bush called Trump-era Republicans "isolationist" and "protectionist" and not "my vision" of the party, the former president walked back some of that criticism in an interview with PEOPLE for next week's issue.

He also freely admitted he did not vote for either incumbent Republican President Donald Trump — of whom he has been only obliquely critical, and never by name — or Democrat Joe Biden in the November election.

Instead, Bush wrote in Condoleezza Rice, who served as his secretary of state from 2005 to 2009.

He told her he did, too.

"She knows it," says Bush, 74, adding wryly: "But she told me she would refuse to accept the office."

• For more on George W. Bush's new project and his life now, subscribe now to PEOPLE or pick up next week's issue.

George W. Bush, Condoleeza Rice
George W. Bush (left), Condoleezza Rice
| Credit: Wesley Hitt/Getty; Noam Galai/Getty

Bush revealed his presidential pick during a wide-ranging conversation with PEOPLE on the publication of his new book of paintings, Out of Many, One, which features a selection of his portraits of American immigrants along with their stories.

It's all in an effort, Bush says, to soften hearts for compassionate immigration reforms after several years of harsh and "frightening" anti-immigrant rhetoric from some in his own party.

According to a news release, Bush's book highlights "the inspiring journeys of America's immigrants and the contributions they make to the life and prosperity of our nation."

The portraits are also on view in an exhibition at the George W. Bush Presidential Center in Dallas.

This is the second book of portraits for Bush, who in 2017 released Portraits of Courage: A Commander in Chief's Tribute to America's Warriorshonoring the military men and women who have served the country since the 9/11 terror attacks that led Bush to start two wars.

George Bush
Credit: Random House

While Bush has largely avoided speaking out about his presidential successors, there have been exceptions: He condemned the violence by Trump supporters during the Jan. 6 insurrection at the U.S. Capitol.

And he has used his art, which started as a hobby, to comment on social and political issues.

In an appearance on the Today show earlier this week, he said of the Republican Party: "I would describe it as isolationist, protectionist and, to a certain extent, nativist."

He clarified those comments to PEOPLE, saying, "Really what I should have said — there's loud voices who are isolationists, protectionists and nativists, something, by the way, I talked about when I was president."

"My concerns [are] about those -isms," he continued, "but I painted with too broad a brush ... because by saying what I said, it excluded a lot of Republicans who believe we can fix the problem."

The "fix" can begin, Bush said, with the Deferred Action for Childhood Arrivals, a policy that allows migrants brought to the U.S. illegally as children to be granted a renewable two-year period of deferred action from deportation and to become eligible for a work permit.

Recipients of the policy are popularly known as "Dreamers," and past attempts at a compromise on immigration reform have included proposals to grant them more permanent status.

Among the portraits in Bush's new book is one of Carlos Mendez.

"I put a DACA student in there because I wanted to just remind Americans that these DACA kids came over when they were... this kid came over when he's 4 years old," Bush tells PEOPLE. "He has no home to go back to. And he's a contributing member of our society, he's engineer and he's smart and he's capable."

During his presidency, Bush unsuccessfully pushed Congress to pass major immigration reforms.

Asked about the controversial wall that was partially built under the Trump administration and remains unfinished, Bush says that the previous president wasn't the only one to build borders between the U.S. and Mexico.

"I built a lot of wall myself, as did Clinton, as did Obama, and the reason why is the American people expect our laws to be upheld," Bush says, harkening back to the tone of compromise when he was in the White House. "But I also believe there's a compassionate way of doing it."

"The truth of the matter is: a wall won't work unless there's comprehensive reform, like work visas, asylum system fixed and border patrol agents focused on their job," Bush says. "And so there needs to be an all-the-above approach to securing the border and we'll see."

He, for one, is optimistic.

"I think, hopefully, once things settle down in Washington they'll be able to focus on this issue and get something done," Bush says, "and maybe this book will help."