Google Similar Image Search Photomosaic

Inspired by this video, I decided to mess around with Google's search by image feature. I took an image, cut it up into tiles, and searched the web for a images similar to each tiny bit. I then stitched the similar images back together. Here's the result, with some more details below.

Here's the finished image, a mosaic of similar images that together approximate Edvard Munch's The Scream. Mouse over for the original.

Here are some details, with technical notes below. Weirdly enough, some monkeys popped up in the results:

And a sagacious orangutan:

A larger version, coming in at 11.6 MB, can be downloaded here. Oddly, there are no naked people or LOLcats, so SafeSearch must be on by default. Or else the internet is broken.

Technical Notes

The original image was cut into 1,008 tiles using ImageMagick:

convert The_Scream.jpg +gravity -crop 32x32 tiles_%d.jpg

Then for each tile, a similar image from the Google search was downloaded with some curl wizardy. They were resized into 200x200 pixel squares with ImageMagick:

convert orig_file.jpg -resize 200x200^ -gravity center \
    -extent 200x200 output_file.png

Then I stiched the similar image tiles together with the ImageMagick mosaic command:

montage 'r_tile%d.png[0-1007]' -tile 28x36 -geometry +0+0 \
    tiles_mosaic.png

The files were named r_tile0.png, r_tile1.png, etc. up to r_tile_1007.png. The images are 28 across by 36 down, and the -geometry +0+0 means no padding space will exist between the images.

I wanted to make a video of the tiles being placed, and I thought I could make the frames with ImageMagick and then put them together into a movie with dvd-slideshow. I wrote a Perl script to loop over the tile images, making a frame for each one, which you can see in the video above. I first made a image of the final mosaic with a black space to the right of it. Onto that image, I drew two black boxes to cover up the tiles that have yet to be drawn. I then paste the tile image into the space on the right. The Perl script, simplified, is pasted at the bottom of this page, but first here are examples of the important ImageMagick commands. First, to black out the image:

convert 480_black.png -fill black -stroke black -strokewidth 0 \
    -draw "rectangle 13,0 373,480" \
    -draw "rectangle 0,13 373,480" \
    blackedout.png

This would draw rectanges with coordinates [(13,0), (373,480)] and [(0,13), (373,480)]. The second coordinate in each pair is the lower right hand corner of the mosaic. This second command simply overlays the tile onto the blacked out image:

composite r_tile0.png -geometry +446+140 \
    blackedout.png frames/frame_0.jpg

It's pasted at (446,140). Below is the Perl script that executes these commands:

for (my $i = 0; $i < 36; $i++) {
	for (my $j = 0; $j < 28; $j++) {

		my $tile_num = ($i * 28) + $j;
		
		# Compute Y of black box 1
		my $bb1_y = int(($i+1) * 13.333333);
		# X of black box 1 is always zero
		my $bb1_x = 0;
		
		# Compute Y of black box 2
		my $bb2_y = int($i * 13.333333);
		# Compute X of black box 2
		my $bb2_x = int(($j+1) * 13.333333);
		
		# Add black boxes to image to cover up 
		#  parts of mosaic
		my $rec_cmd = 'convert 480_black.png -fill black ';
		$rec_cmd .= '-stroke black -strokewidth 0 ';
		$rec_cmd .= '-draw "rectangle ' . $bb1_x . ',';
		$rec_cmd .= $bb1_y . ' 373,480" ';
		$rec_cmd .= '-draw "rectangle ' . $bb2_x . ',';
		$rec_cmd .= $bb2_y . ' 373,480" ';
		$rec_cmd .= ' blackedout.png';
		
		system($rec_cmd);
		
		# Add tile image overlay
		my $comp_cmd = 'composite ../renumbered/r_tile';
		$comp_cmd .= $tile_num . '.png -geometry +446+140 ';
		$comp_cmd .= '480_blackedout.png frames/frame_';
		$comd_cmd .= $tile_num . '.jpg';
		
		system($comp_cmd);	
	}
}

Then I used dvd-slideshow to put the entire thing together, and added the audio found through a Creative Commons music search site. The song, "I dunno" by grape, can be found here:

grapes / CC BY 3.0

Now time to start studying for comps...