The affable contestant didn't put up a fuss – and it cost him the game
Credit: Monty Brinton/CBS

In the end, he was just too nice.

Jeremiah Wood was voted off Survivor: Cagayan on Day 28 after finding himself down in numbers. When Jefra Bland told him that she wouldn’t work with him, Wood accepted the inevitable rather than causing a scene – a move he now regrets.

The 34-year-old model from Dobson, N.C., tells PEOPLE where he went wrong – and why he let his looks go before going on the show.

Jeremiah, in your words, what happened?
It was as simple as this: I just didn’t have the numbers.

But the last episode made it look like you might convince Jefra to flip. Was that the case?
Yeah, pretty much what you saw is what happened. We flipped her, but then Trish flipped her back.

Jefra came to you and said she wasn’t going to work with you. You were like, ‘Okay, that’s all right. Thanks for telling me.’ Why weren’t you more aggressive?
That was a big mistake. I kick myself over that. I think I played a good game up to that. I should have done something big; made up a big lie or really confronted her. But she did it in front of everybody.

Jefra seems like the type of person who hates conflict. You basically gave her the okay to vote against you, without feeling bad about it.
She’s an absolute sweetheart. If we had been by ourselves, I would have pleaded with her and told her where she was messing up. But when she was in front of everyone, I knew it was going to be tough. If I had confronted her, she would have started to cry in front of everyone, and then there would have been even a bigger target on me.

Let her cry! It’s Survivor!
[Laughs] I know, but I really didn’t want to be mean about it. She had made her decision, and it sealed my fate.

Why didn’t she trust you? You were aligned at the beginning of the game.
I shook hands with Jefra on the third or fourth day and told her I’d always be with her. After the switch, I knew she had flipped on me. When I voted for her after the merge, she held a grudge. I apologized a million times, told her that I was trying to win, but she didn’t trust me anymore.

At tribal council, Tony pulled out an idol and then said it was fake. We all know now that it was a real idol. What did you think at the time?
I didn’t know. When you’re hungry and hot and tired, you’re not thinking straight. So we didn’t know what it meant.

You should have told him to prove it by throwing it in the fire.
Oh, man! I should have. That would have been the right play, to force him to admit that it was a real idol! But again, things were crazy. I wasn’t thinking straight.”

You had a big reveal that you’re actually a working fashion model. Why did you keep that a secret?
I kept it a secret because I wanted to prove that I was not a model who wanted TV time to promote modeling or acting. I was there to play the game. So I told them that I remodel homes, which is what I do when I’m not doing the modeling thing. I wanted everyone to believe that I was an old country boy who worked his butt off.

So were you on Survivor to further an acting career?
Not at all! Survivor is one of my favorite shows, and I really wanted to play the game. I had done a few audition tapes. This time, they finally chose me! I didn’t want to look like a model. I gained weight and let my hair go. I was simply there because I love the game so much.