James was born in Wexford, Ireland, in 1847, and he emigrated to the United States, where he enlisted in the Army. He deployed to the Arizona Territory as a Sergeant with Company F of the 5th US Cavalry in 1872 and it was his actions on the 27th of August 1872, that would earn him the Medal of Honor. The citation reads:

In command of a detachment of four men, defeated a superior force.

James Brown received the Medal of Honor on the 4th of December 1874, and the details of his later life and death are unknown. He is one of 314 recipients that are considered to be lost to history.