This should help. The documentary Lost Duluth (available on Youtube) uses a photo that is cropped differently but must have been taken at almost the exact same time and place as our mystery photo (all the construction debris in the street matches up, but the angle looking down the street is just slightly different). The documentary uses use the photo to illustrate the state of Superior Street at the start of the population boom in 1869 (although the documentary never states the precise year of the photo). The Clark House was built in 1870, so it could reasonably be from just before construction started. The photo used in the documentary is a bit sharper and may contain information that could help us know more about both photos.
 

I think everyone who commented above really solved this, but taking everyone's suggestions together seems to give a rather clear answer. 



Using Tony's suggestion of looking in the MNreflection archives, this large panorama photo from 1870 clearly shows the area of the mystery photo on the right with the Clark House several blocks to the west.



Another photo labeled as "Superior Street buildings at First Avenue east from Minnesota Point, Duluth, Minnesota" from June of 1870 shows many of the same buildings from the mystery photo at a different angle, but with a much lower building density than we can see in the mystery photo.



This photo from around 1873 seems to have about the same building density with a clear image of the old First Presbyterian Church at 231 E. Second St., visible in the background of the mystery photo as stated by Roger. 



Finally, the photo used in the Lost Duluth documentary that must have been taken on the same day because all the construction material in the street is in the exact same position in both photos shows the relationship of the street to the railroad tracks and harbor.

All of which suggests the mystery photo was taken looking east down Superior street toward First Avenue East around 1873.