John was born on the 3rd of January, 1876 in Brewster, New York, and joined the Merchant Marines at the age of 15. Seven years later, he enlisted in the US Navy and served on the USS Columbia during the Spanish-American War in the West Indies. John then served in the China Relief Expedition, which rescued US citizens, European, and other foreign nationals from China during the late years of the Boxer Rebellion. It was his actions during this expedition that would earn him his first Medal of Honor. The citation reads:

In action with the relief expedition of the Allied forces in China, 13, 20, 21, and 22 June 1900. During this period and in the presence of the enemy, Coxswain McCloy distinguished himself by meritorious conduct.

John was then deployed to support the Mexican Revolution and it was his actions while in charge of the unloading of men and supplies at the pier that would earn him his second Medal of Honor. The citation reads:

For heroism in leading 3 picket launches along Vera Cruz sea front, drawing Mexican fire and enabling cruisers to save our men on shore, April 22, 1914. Though wounded, he gallantly remained at his post.

John had maneuvered his boat away from the men at the pier to draw fire from a detachment from the Mexican Naval Academy and this exposed the enemy's positions so that other cruisers could locate and shell their positions. He had also been shot in the thigh but did not leave his post until ordered to the hospital by the brigade surgeon.

John went on to serve in World War 1 as an Ensign but what exactly he did during the war isn't clear. Following the war, he became the first commander of the USS Curlew, which was a minesweeper used in the North Sea. For his work clearing the mine barrage, John received the Navy Cross and was promoted to Lieutenant. He went on to command another minesweeper, the USS Lark, from Massachusetts and then the USS Dobbin. John retired on the 15th of October, 1928 after thirty years of service and was active in the Veterans of Foreign War and the American Legion. He was a founding member of the Legion's first post in the state of New Jersey, what is now known as the SGT Clendenon S Newell Post #1 in Leonia. John was later promoted in retirement to Lieutenant Commander on the 23rd of February, 1942, and much later, in 2010, the United States Postal Service issued a series of Distinguished Sailors stamps with one having his portrait. John McCloy died in his home in Leonia on the 25th of May, 1945, at the age of 69, and is buried with his wife Sarah in Arlington National Cemetery: Section 8, Lot 5246.