James was born on the 12th of January 1919, in Wilmington, Delaware, and when he was 3, his mother died, and he and his sister went to live with their aunt. He was active in, and excelled at, basketball and football while at St Mary’s Parochial School, but he left school after the eighth grade to work for a local leather company. James was drafted into the US Army in January of 1941 but instead of being discharged at the end of the year, his contact was extended after the attacks on Pearl Harbor. He was deployed to north Africa and Europe, and it was his actions with the 7th Infantry Regiment, 3rd Infantry Division, that would earn him the Medal of Honor. The citation reads:

For conspicuous gallantry and intrepidity at risk of life above and beyond the call of duty. On 15 August 1944 Sgt. Connor, through sheer grit and determination, led his platoon in clearing an enemy vastly superior in numbers and firepower from strongly entrenched positions on Cape Cavalaire, removing a grave enemy threat to his division during the amphibious landing in southern France, and thereby insured safe and uninterrupted landings for the huge volume of men and material which followed. His battle patrol landed on "Red Beach" with the mission of destroying the strongly fortified enemy positions on Cape Cavalaire with utmost speed. From the penninsula the enemy had commanding observation and seriously menaced the vast landing operations taking place. Though knocked down and seriously wounded in the neck by a hanging mine which killed his platoon lieutenant, Sgt. Connor refused medical aid and with his driving spirit practically carried the platoon across several thousand yards of mine-saturated beach through intense fire from mortars, 20-mm flak guns, machine guns, and snipers. En route to the cape he personally shot and killed two snipers. The platoon sergeant was killed and Sgt. Connor became platoon leader. Receiving a second wound, which lacerated his shoulder and back, he again refused evacuation, expressing determination to carry on until physically unable to continue. He reassured and prodded the hesitating men of his decimated platoon forward through almost impregnable mortar concentrations. Again emphasizing the prevalent urgency of the mission, he impelled his men toward a group of buildings honeycombed with enenmy snipers and machine guns. Here he received his third grave wound, this time in the leg, felling him in his tracks. Still resolved to carry on, he relinquished command only after his attempts proved that it was physically impossible to stand. Nevertheless, from his prone position, he gave the orders and directed his men in assualting the enemy. Infused with Sgt. Connor's dogged determination, the platoon, though reduced to less than one-third of its original 36 men, outflanked and rushed the enemy with such furiousness that they killed seven, captured 40, seized three machine guns and considerable other material, and took all their assigned objectives, successfully completing their mission. By his repeated examples of tenaciousess and indomitable spirit Sgt. Connor transmitted his heroism to his men until they became a fighting team which could not be stopped.

James was sent to an Italian hospital to recover from his wounds and was mistakenly sent back to the front lines to fight, which lasted for a few days before being sent home instead. He later recalled that going back to the front lines and seeing the destruction of the men he had fought with was the hardest thing he ever had to do. Before leaving Europe, James received the Medal of Honor from Lieutenant General Alexander Patch, and on the 7th of May 1945, just five days after the effective surrender of the Germans, he traveled to the White House to be honored by President Truman.

James was married to Elizabeth shortly after returning home and the couple had four sons. He went on to work for the Veterans Administration for 34 years in Delaware, and on the 27th of July 1994, James Phillip Connor, died of heart failure at the age of 75. He is buried with his wife in the Delaware Veterans Memorial Cemetery, in Bear, Delaware: Section 1, Field B, Row C, Grave 22.