Judge Clough died in La Mesa, Calif., in 1910 at the age of 82. He was a pioneer of Superior, served eight years on the local circuit court bench starting circa 1870, serving three terms, thence on to Wis. state supreme court. J. C. Spooner was his law partner for a time (who became a U.S. Senator), as was Colonel Hiram Hayes. 

He left Superior in 1895, at which time he was president and director at the First National Bank of Superior, which he helped found. He owned a lot more than that island on the waterfront. One of his daughters married Irvine Lenroot (also a U.S. Senator from Wisconsin), who also nearly became vice president under Warren J. Harding (1920 -- he delayed his decision too long, and Calvin Coolidge supporters moved in, and since Harding died in office, Coolidge became prez). 

You can read about the Lenroot family here.

There has always been a lot of controversy about Clough Island. In recent times it was about ecology/preservation. But in times past, there were issues over it being Wisconsin territory despite the fact that it's so close to the Minnesota side. There was a court case about it in 1917 (mostly about who got the taxes for some docks that were debated), and Alfred Merritt told the court that "it was a well known tradition that George B. Stuntz of Superior was employed by the federal government to survey the boundary line he admitted that he rowed his canoe so as to get as much land for Wisconsin as possible." 

Circa 1897 there were plans to build an armor plate factory on the island. 

There's a booklet at the library by Claire Schumacher called "The Whiteside Island Story" -- though some local historians are skeptical of some of her stories. 

Robert Whiteside had a farm there that was 200 acres, growing oats, potatoes, peas and hay. There's also a story that after the family left the farm behind, they left this big white farm horse, who lived there by himself for some years, running free on the meadows.

E.P. Alexander had 10 acres on the property, and during Prohibition, there are rumors of bootlegging, prostitution and gambling. 

You can read about E.P. Alexander and his Killjoy Lodge here.