During the afternoon truce, Cole confirmed for his comrade Ben Robinson that the leader of the Confederate attackers was indeed Forrest. Cole, a native of Tennessee, had been sold by Forrest once in his life. When Robinson asked what the conditions were like, Cole admitted that comparatively, they weren't that bad: the "holding pen" was not overcrowded, the slaves were given beds to sleep on and were fed enough.
From their vantage point on the parapet, Robinson and Cole could see Forrest meeting with Union troops. Robinson lamented the fact that the truce was on, as it would have been a golden opportunity to kill Forrest. Cole reminded Robinson that they should obey legal nicities. If they did kill Forrest, the the lives of every black troop would have been forfeit if the Confederates made it in. Even now, there was a strong probability that the Confederates would give no quarter. Robinson pointed out that so long as they were armed, they could make the Cofederates pay.
When the attack finally came, Robinson was the senior officer command his gun, after Sgt. Clark was injured. Robinson was able to use his gun on attackers at point blank range. After that single shot, however, there wasn't enough time to reload. Cole, Robinson, and Key used the various tools for cleaning and reloading the gun as weapons, keeping the Confederates at bay, but the three were forced to retreat after grabbing discarded muskets.
The trio were joined by Nathan Hunter and Aaron Fentis in a small group who sought desperately to fend off the attackers. To an extent, they were able to do so. But Cole saw that Confederate troops had gotten into the fort behind them. The group fled to the banks of the Mississippi River, where the found stores of ammunition placed on the river by Major William Bradford before the battle began in earnest. As the group began reloading their weapons, they saw Confederate troops were firing on the USS New Era, a ship that had been providing artillery cover during the battle, and was supposed to be covering the retreat of the garrison. Confederate sharpshooters were not letting the gunners open up their hatches. Finally, the New Era steamed up-river.
Things went from bad to worse for the group, as each were injured in turn, although none were fatally so. Cole himself was shot in the right thigh and arm. When the fighting was over, Cole was among those picked up during a truce by Union forces, and taken to a hospital in Mound City, Illinois, to recover. Here, he gave testimony to Senator Benjamin Wade and Congressman Daniel Gooch about the battle.