The exchange, which was captured on video by Washington Post reporter Steven Zeitchik, took place during a panel moderated by Oliver for a 20th anniversary screening of “Wag the Dog,” which starred Hoffman.
Oliver was referring to recent allegations made by Anna Graham Hunter in an essay, which was published on November 1 by The Hollywood Reporter.
In the column, she claims Hoffman, now 80, made multiple inappropriate comments and unwanted sexual advances toward her when she was a 17-year-old production assistant on the set of “The Death of a Salesman” in 1985.
“From a few things you’ve read you’ve made an incredible assumption about me. You’ve made the case better than anyone else can. I’m guilty,” he told Oliver.
On Monday, Hoffman duked it out with Oliver on the stage, where the actor was joined by his fellow “Wag the Dog” co-star Robert De Niro, producer Jane Rosenthal and director Barry Levinson.
In the video, Oliver called out Hoffman’s apology, saying he was angered by the actor’s word choice.
Hoffman also questioned whether Oliver believes everything printed about what his accuser says happened.
Hoffman described how his co-workers were “a family,” who sometimes said stupid sexual things, that were meant to be funny.
CNN has reached out to Hoffman and Oliver’s reps for comment.