Ghosts of Animals Haunt Portland’s Overpriced Apartments

[This article was first published on Dan Garmat's Blog -- R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Apartment hunting in an expensive city is leading me to curses and exclamations. Following are some outstanding examples of insanely priced apartments in Portland, OR, run through Google Deep Dream in hopes of my understanding why people pay so much for a small box. These listings will be gone in no time (I’m sure) so including some captions for posterity.

Let’s start with this one. Indeed, it appears $1899 for 1 bedroom grants access to this clubhouse haunted by some floating apparition:

clubhousedd

Deep Dream InceptionV3 algorithm here is trained on ImageNet, then makes changes that increase confidence in the predicted category. Looped several times with the num_octave hyperparameter, it starts to look a good bit trippy and helps give some intuition what a neural network “sees” as prototypical examples of a predicted class. Apparently there is no “view of apartment” class as it keeps seeing ghastly animals. Perhaps it is no coincidence even before running InceptionV3 this clubhouse already looks like it could work in The Shining:

clubhouseorig

$1850 / 1br – 697ft2 – BRAND NEW! Enjoy Luxury Uban [Urban?] Living at The Franklin Flats!

“NEW ON THE MARKET!

“The Franklin Flats is the newest addition to this desirable part of town! Built with the urban adventurer in mind, our small community offers luxury appeal with a neighborhood feel. Boasting a walkability score of 86 out of 100, you can’t beat the location! [unless an 87+?] Our close proximity to Mt. Tabor, food carts, [because you won’t have anything left over for restaurants] shopping and eateries gives you the classic Northwest experience you crave. Email or call to schedule a personal tour today!”

franklin01

Apparently the Attack on Titan seals make this a desirable part of town.

Perhaps those seals are why walkability doesn’t crack 90. If you survive the seals on a midday stroll, there are titan polar bears who can walk through walls:

franklin02

$4250 / 2br – 1900ft2 – Condo on the Willamette

“Breathtaking views of the city and the Willamette River, located in the elegant Atwater. This condo has two bedrooms, living room, dining room, gourmet kitchen, gas fireplace, small office, two balconies, utility room and underground parking. Includes concierge desk, card-accessed security.”

Something tells me this view of the Willamette River would be complete if a cocker spaniel is staring at me…

atwater01

But this view is what you really pay for: look at all the suckers in the two identical buildings who massively overpaid for – how the heck did those get up here?!

atwater02

$3900 / 2br – 1004ft2 – Portland’s most well-appointed two bedroom apartments now available

“Portland’s premier rental community is now pre-leasing. Find everything you desire in one place: The finest dining, distinguished boutiques, and most-beloved haunts. Experience the perfect merger of luxury and livability with our timeless brick exterior, stunning marble lobby, tiled bathrooms, tall ceilings, ample light, extensive storage, concierge services, and even a dog washing station.

“Proudly situated in Portland’s [S]Nob Hill/23rd Ave neighborhood, 21 Astor boasts a “97” walk score and a “96” bike score. [Beat that Franklin Flats!] Life is great in the epicenter of Portland’s charm. Pick up your locally-grown produce at City Market. Grab happy hour with the gang at Seratto. Sweat it out at Corepower Yoga.

“Our greater than 1:1 parking ratio assures a space for your car in our garage. Imagine never having to find parking on NW 23rd again.”

I work on NW 21st, that’s almost the cost of parking alone. People walk by on Oregon ‘pot tours’ and this may be how they see the building as well:

well-appointed01

Whoa! Is that a pig in the sky? Far out!

At least their new $3.88/sqft/month kitchens aren’t yet haunted – oh, god! Are those giant psychedelic worms!

well-appointed02

Code:

Started with JJ Allaire, François Chollet, RStudio, and Google’s keras code based on Google DeepDream. Added a little looping to try different parameters over these 7 images:

<span class="n">library</span><span class="p">(</span><span class="n">keras</span><span class="p">)</span><span class="w">
</span><span class="n">library</span><span class="p">(</span><span class="n">tensorflow</span><span class="p">)</span><span class="w">
</span><span class="n">library</span><span class="p">(</span><span class="n">purrr</span><span class="p">)</span><span class="w">

</span><span class="c1"># Function Definitions ----------------------------------------------------</span><span class="w">

</span><span class="n">preprocess_image</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">image_path</span><span class="p">){</span><span class="w">
  </span><span class="n">image_load</span><span class="p">(</span><span class="n">image_path</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w">
    </span><span class="n">image_to_array</span><span class="p">()</span><span class="w"> </span><span class="o">%>%</span><span class="w">
    </span><span class="n">array_reshape</span><span class="p">(</span><span class="n">dim</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">c</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="w"> </span><span class="nf">dim</span><span class="p">(</span><span class="n">.</span><span class="p">)))</span><span class="w"> </span><span class="o">%>%</span><span class="w">
    </span><span class="n">inception_v3_preprocess_input</span><span class="p">()</span><span class="w">
</span><span class="p">}</span><span class="w">

</span><span class="n">deprocess_image</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">x</span><span class="p">){</span><span class="w">
  </span><span class="n">x</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">x</span><span class="p">[</span><span class="m">1</span><span class="p">,,,]</span><span class="w">
  
  </span><span class="c1"># Remove zero-center by mean pixel</span><span class="w">
  </span><span class="n">x</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">x</span><span class="o">/</span><span class="m">2</span><span class="n">.</span><span class="w">
  </span><span class="n">x</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="m">0.5</span><span class="w">
  </span><span class="n">x</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="m">255</span><span class="w">
  
  </span><span class="c1"># 'BGR'->'RGB'</span><span class="w">
  </span><span class="n">x</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">x</span><span class="p">[,,</span><span class="nf">c</span><span class="p">(</span><span class="m">3</span><span class="p">,</span><span class="m">2</span><span class="p">,</span><span class="m">1</span><span class="p">)]</span><span class="w">
  
  </span><span class="c1"># Clip to interval 0, 255</span><span class="w">
  </span><span class="n">x</span><span class="p">[</span><span class="n">x</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="m">255</span><span class="p">]</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="m">255</span><span class="w">
  </span><span class="n">x</span><span class="p">[</span><span class="n">x</span><span class="w"> </span><span class="o"><</span><span class="w"> </span><span class="m">0</span><span class="p">]</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="m">0</span><span class="w">
  </span><span class="n">x</span><span class="p">[]</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">as.integer</span><span class="p">(</span><span class="n">x</span><span class="p">)</span><span class="o">/</span><span class="m">255</span><span class="w">
  </span><span class="n">x</span><span class="w">
</span><span class="p">}</span><span class="w">

</span><span class="c1"># Parameters --------------------------------------------------------</span><span class="w">

</span><span class="c1">## list of images to process --</span><span class="w">
</span><span class="n">list_images</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">list.files</span><span class="p">(</span><span class="s1">'images/deep dream apartments/orig/'</span><span class="p">,</span><span class="w"> </span><span class="n">full.names</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">TRUE</span><span class="p">)</span><span class="w">

</span><span class="c1">## list of settings to try --</span><span class="w">
</span><span class="n">list_settings</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
  </span><span class="n">settings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
    </span><span class="n">features</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">mixed2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.2</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed3</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.5</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed4</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2</span><span class="n">.</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed5</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.5</span><span class="p">),</span><span class="w">
    </span><span class="n">hyperparams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="c1"># Playing with these hyperparameters will also allow you to achieve new effects</span><span class="w">
      </span><span class="n">step</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.01</span><span class="p">,</span><span class="w">  </span><span class="c1"># Gradient ascent step size</span><span class="w">
      </span><span class="n">num_octave</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">6</span><span class="p">,</span><span class="w">  </span><span class="c1"># Number of scales at which to run gradient ascent</span><span class="w">
      </span><span class="n">octave_scale</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.4</span><span class="p">,</span><span class="w">  </span><span class="c1"># Size ratio between scales</span><span class="w">
      </span><span class="n">iterations</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">20</span><span class="p">,</span><span class="w">  </span><span class="c1"># Number of ascent steps per scale</span><span class="w">
      </span><span class="n">max_loss</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">10</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">),</span><span class="w">
  </span><span class="n">settings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
    </span><span class="n">features</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">mixed2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.5</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed3</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.2</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed4</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.1</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed5</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.5</span><span class="p">),</span><span class="w">
    </span><span class="n">hyperparams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">step</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.01</span><span class="p">,</span><span class="w">  
      </span><span class="n">num_octave</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">9</span><span class="p">,</span><span class="w"> 
      </span><span class="n">octave_scale</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.1</span><span class="p">,</span><span class="w">  
      </span><span class="n">iterations</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">20</span><span class="p">,</span><span class="w">  
      </span><span class="n">max_loss</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">5</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">),</span><span class="w">
  </span><span class="n">settings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
    </span><span class="n">features</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">mixed2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.02</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed3</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.05</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed4</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.01</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed5</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.05</span><span class="p">),</span><span class="w">
    </span><span class="n">hyperparams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">step</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.01</span><span class="p">,</span><span class="w">  
      </span><span class="n">num_octave</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">11</span><span class="p">,</span><span class="w"> 
      </span><span class="n">octave_scale</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.1</span><span class="p">,</span><span class="w">  
      </span><span class="n">iterations</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">20</span><span class="p">,</span><span class="w">  
      </span><span class="n">max_loss</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">20</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">),</span><span class="w">
  </span><span class="n">settings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
    </span><span class="n">features</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">mixed2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.2</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed3</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.5</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed4</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2</span><span class="n">.</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed5</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.5</span><span class="p">),</span><span class="w">
    </span><span class="n">hyperparams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">step</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.01</span><span class="p">,</span><span class="w">  
      </span><span class="n">num_octave</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">8</span><span class="p">,</span><span class="w"> 
      </span><span class="n">octave_scale</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.4</span><span class="p">,</span><span class="w">  
      </span><span class="n">iterations</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">20</span><span class="p">,</span><span class="w">  
      </span><span class="n">max_loss</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">10</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">),</span><span class="w">
  </span><span class="n">settings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
    </span><span class="n">features</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">mixed2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.2</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed3</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.1</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed4</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.4</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed5</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.3</span><span class="p">),</span><span class="w">
    </span><span class="n">hyperparams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">step</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.01</span><span class="p">,</span><span class="w">  
      </span><span class="n">num_octave</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">8</span><span class="p">,</span><span class="w"> 
      </span><span class="n">octave_scale</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.4</span><span class="p">,</span><span class="w">  
      </span><span class="n">iterations</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">20</span><span class="p">,</span><span class="w">  
      </span><span class="n">max_loss</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">10</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">),</span><span class="w">
  </span><span class="n">settings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
    </span><span class="n">features</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">mixed2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.2</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed3</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.5</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed4</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">3</span><span class="n">.</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed5</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2.5</span><span class="p">),</span><span class="w">
    </span><span class="n">hyperparams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">step</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.01</span><span class="p">,</span><span class="w">  
      </span><span class="n">num_octave</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">8</span><span class="p">,</span><span class="w"> 
      </span><span class="n">octave_scale</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.4</span><span class="p">,</span><span class="w">  
      </span><span class="n">iterations</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">20</span><span class="p">,</span><span class="w">  
      </span><span class="n">max_loss</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">25</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">),</span><span class="w">
  </span><span class="n">settings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
    </span><span class="n">features</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">mixed2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.2</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed3</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2.5</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed4</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2</span><span class="n">.</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed5</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">3.5</span><span class="p">),</span><span class="w">
    </span><span class="n">hyperparams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">step</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.05</span><span class="p">,</span><span class="w">  
      </span><span class="n">num_octave</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">6</span><span class="p">,</span><span class="w"> 
      </span><span class="n">octave_scale</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.4</span><span class="p">,</span><span class="w">  
      </span><span class="n">iterations</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">20</span><span class="p">,</span><span class="w">  
      </span><span class="n">max_loss</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">13</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">),</span><span class="w">
  </span><span class="n">settings</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
    </span><span class="n">features</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">mixed2</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.2</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed3</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2.5</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed4</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">2</span><span class="n">.</span><span class="p">,</span><span class="w">
      </span><span class="n">mixed5</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">3.5</span><span class="p">),</span><span class="w">
    </span><span class="n">hyperparams</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="w">
      </span><span class="n">step</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">0.05</span><span class="p">,</span><span class="w">  
      </span><span class="n">num_octave</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">8</span><span class="p">,</span><span class="w"> 
      </span><span class="n">octave_scale</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">1.4</span><span class="p">,</span><span class="w">  
      </span><span class="n">iterations</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">20</span><span class="p">,</span><span class="w">  
      </span><span class="n">max_loss</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="m">13</span><span class="w">
    </span><span class="p">)</span><span class="w">
  </span><span class="p">)</span><span class="w">
</span><span class="p">)</span><span class="w">

</span><span class="c1"># Model Definition --------------------------------------------------------</span><span class="w">

</span><span class="n">k_set_learning_phase</span><span class="p">(</span><span class="m">0</span><span class="p">)</span><span class="w">

</span><span class="c1"># Build the InceptionV3 network with our placeholder.</span><span class="w">
</span><span class="c1"># The model will be loaded with pre-trained ImageNet weights.</span><span class="w">
</span><span class="n">model</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">application_inception_v3</span><span class="p">(</span><span class="n">weights</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"imagenet"</span><span class="p">,</span><span class="w"> </span><span class="n">include_top</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">FALSE</span><span class="p">)</span><span class="w">

</span><span class="c1"># This will contain our generated image</span><span class="w">
</span><span class="n">dream</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">model</span><span class="o">$</span><span class="n">input</span><span class="w">

</span><span class="c1"># Get the symbolic outputs of each "key" layer (we gave them unique names).</span><span class="w">
</span><span class="n">layer_dict</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">model</span><span class="o">$</span><span class="n">layers</span><span class="w">
</span><span class="nf">names</span><span class="p">(</span><span class="n">layer_dict</span><span class="p">)</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">map_chr</span><span class="p">(</span><span class="n">layer_dict</span><span class="w"> </span><span class="p">,</span><span class="o">~</span><span class="n">.x</span><span class="o">$</span><span class="n">name</span><span class="p">)</span><span class="w">


</span><span class="c1">## just loop on images - bottleneck is training anyway --</span><span class="w">
</span><span class="k">for</span><span class="p">(</span><span class="n">image_path</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="n">list_images</span><span class="p">){</span><span class="w">
  
  </span><span class="n">image</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">preprocess_image</span><span class="p">(</span><span class="n">image_path</span><span class="p">)</span><span class="w">
  
  </span><span class="c1">#res <- predict(model,image)</span><span class="w">
  </span><span class="c1">#reduce(dim(res)[-1], .f = `*`) %>% </span><span class="w">
  </span><span class="c1">#  as.numeric() %>% </span><span class="w">
  </span><span class="c1">#array_reshape(res, c(2, 51200)) %>% </span><span class="w">
  </span><span class="c1">#  imagenet_decode_predictions()</span><span class="w">
  </span><span class="c1">## get model prediction:</span><span class="w">
  </span><span class="c1">#x <- image_to_array(image)</span><span class="w">
  </span><span class="c1">#x <- array_reshape(image, c(1, dim(image)))</span><span class="w">
  </span><span class="c1">#x <- imagenet_preprocess_input(image)</span><span class="w">
  </span><span class="c1">#preds = predict(model, x)</span><span class="w">
  </span><span class="c1">#imagenet_decode_predictions(preds, top = 3)</span><span class="w">
  </span><span class="c1">#predict_classes(model, image)</span><span class="w">
  </span><span class="c1"># need to try again later</span><span class="w">
  
  </span><span class="n">setting_counter</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="m">0</span><span class="w">
  
  </span><span class="c1">## then nested loop on settings --</span><span class="w">
  </span><span class="k">for</span><span class="p">(</span><span class="n">settings</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="n">list_settings</span><span class="p">){</span><span class="w">
    </span><span class="n">setting_counter</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">setting_counter</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="m">1</span><span class="w">
    </span><span class="n">print</span><span class="p">(</span><span class="n">paste0</span><span class="p">(</span><span class="s1">'starting '</span><span class="p">,</span><span class="w"> </span><span class="n">image_path</span><span class="p">,</span><span class="w"> </span><span class="s1">', setting # '</span><span class="p">,</span><span class="w">  </span><span class="n">setting_counter</span><span class="p">))</span><span class="w">
    
    </span><span class="c1"># reload image each time</span><span class="w">
    </span><span class="n">image</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">preprocess_image</span><span class="p">(</span><span class="n">image_path</span><span class="p">)</span><span class="w">
    
    
    </span><span class="c1"># Define the loss</span><span class="w">
    </span><span class="n">loss</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">k_variable</span><span class="p">(</span><span class="m">0.0</span><span class="p">)</span><span class="w">
    </span><span class="k">for</span><span class="p">(</span><span class="n">layer_name</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="nf">names</span><span class="p">(</span><span class="n">settings</span><span class="o">$</span><span class="n">features</span><span class="p">)){</span><span class="w">
      
      </span><span class="c1"># Add the L2 norm of the features of a layer to the loss</span><span class="w">
      </span><span class="n">coeff</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">settings</span><span class="o">$</span><span class="n">features</span><span class="p">[[</span><span class="n">layer_name</span><span class="p">]]</span><span class="w">
      </span><span class="n">x</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">layer_dict</span><span class="p">[[</span><span class="n">layer_name</span><span class="p">]]</span><span class="o">$</span><span class="n">output</span><span class="w">
      </span><span class="n">scaling</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">k_prod</span><span class="p">(</span><span class="n">k_cast</span><span class="p">(</span><span class="n">k_shape</span><span class="p">(</span><span class="n">x</span><span class="p">),</span><span class="w"> </span><span class="s1">'float32'</span><span class="p">))</span><span class="w">
      
      </span><span class="c1"># Avoid border artifacts by only involving non-border pixels in the loss</span><span class="w">
      </span><span class="n">loss</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">loss</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">coeff</span><span class="o">*</span><span class="n">k_sum</span><span class="p">(</span><span class="n">k_square</span><span class="p">(</span><span class="n">x</span><span class="p">))</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="n">scaling</span><span class="w">
    </span><span class="p">}</span><span class="w">
    
    
    </span><span class="c1"># Compute the gradients of the dream wrt the loss</span><span class="w">
    </span><span class="n">grads</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">k_gradients</span><span class="p">(</span><span class="n">loss</span><span class="p">,</span><span class="w"> </span><span class="n">dream</span><span class="p">)[[</span><span class="m">1</span><span class="p">]]</span><span class="w"> 
    
    </span><span class="c1"># Normalize gradients.</span><span class="w">
    </span><span class="n">grads</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">grads</span><span class="w"> </span><span class="o">/</span><span class="w"> </span><span class="n">k_maximum</span><span class="p">(</span><span class="n">k_mean</span><span class="p">(</span><span class="n">k_abs</span><span class="p">(</span><span class="n">grads</span><span class="p">)),</span><span class="w"> </span><span class="n">k_epsilon</span><span class="p">())</span><span class="w">
    
    </span><span class="c1"># Set up function to retrieve the value</span><span class="w">
    </span><span class="c1"># of the loss and gradients given an input image.</span><span class="w">
    </span><span class="n">fetch_loss_and_grads</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">k_function</span><span class="p">(</span><span class="nf">list</span><span class="p">(</span><span class="n">dream</span><span class="p">),</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="n">loss</span><span class="p">,</span><span class="n">grads</span><span class="p">))</span><span class="w">
    </span><span class="c1"># this function will crash the R session if too many octaves on too small an image</span><span class="w">
    
    </span><span class="n">eval_loss_and_grads</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">image</span><span class="p">){</span><span class="w">
      </span><span class="n">outs</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">fetch_loss_and_grads</span><span class="p">(</span><span class="nf">list</span><span class="p">(</span><span class="n">image</span><span class="p">))</span><span class="w">
      </span><span class="nf">list</span><span class="p">(</span><span class="w">
        </span><span class="n">loss_value</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">outs</span><span class="p">[[</span><span class="m">1</span><span class="p">]],</span><span class="w">
        </span><span class="n">grad_values</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">outs</span><span class="p">[[</span><span class="m">2</span><span class="p">]]</span><span class="w">
      </span><span class="p">)</span><span class="w">
    </span><span class="p">}</span><span class="w">
    
    
    </span><span class="n">gradient_ascent</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">x</span><span class="p">,</span><span class="w"> </span><span class="n">iterations</span><span class="p">,</span><span class="w"> </span><span class="n">step</span><span class="p">,</span><span class="w"> </span><span class="n">max_loss</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kc">NULL</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
      </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">i</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">1</span><span class="o">:</span><span class="n">iterations</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
        </span><span class="n">out</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">tryCatch</span><span class="p">(</span><span class="n">eval_loss_and_grads</span><span class="p">(</span><span class="n">x</span><span class="p">),</span><span class="w"> </span><span class="n">error</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">function</span><span class="p">(</span><span class="n">e</span><span class="p">)</span><span class="w"> </span><span class="kc">NA</span><span class="p">)</span><span class="w"> </span><span class="c1"># need to add this for negative gradients</span><span class="w">
        </span><span class="k">if</span><span class="p">(</span><span class="nf">is.na</span><span class="p">(</span><span class="n">out</span><span class="o">$</span><span class="n">loss_value</span><span class="p">)){</span><span class="w">
          </span><span class="n">print</span><span class="p">(</span><span class="n">paste0</span><span class="p">(</span><span class="s1">'NA loss_value on setting # '</span><span class="p">,</span><span class="w"> </span><span class="n">setting_counter</span><span class="p">))</span><span class="w">
          </span><span class="k">break</span><span class="w">
        </span><span class="p">}</span><span class="w"> </span><span class="k">else</span><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="p">(</span><span class="o">!</span><span class="nf">is.null</span><span class="p">(</span><span class="n">max_loss</span><span class="p">)</span><span class="w"> </span><span class="o">&</span><span class="w"> </span><span class="n">out</span><span class="o">$</span><span class="n">loss_value</span><span class="w"> </span><span class="o">></span><span class="w"> </span><span class="n">max_loss</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
          </span><span class="k">break</span><span class="w">
        </span><span class="p">}</span><span class="w"> 
        </span><span class="n">print</span><span class="p">(</span><span class="n">paste</span><span class="p">(</span><span class="s2">"Loss value at"</span><span class="p">,</span><span class="w"> </span><span class="n">i</span><span class="p">,</span><span class="w"> </span><span class="s1">':'</span><span class="p">,</span><span class="w"> </span><span class="n">out</span><span class="o">$</span><span class="n">loss_value</span><span class="p">))</span><span class="w">
        </span><span class="n">x</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">x</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">step</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="n">out</span><span class="o">$</span><span class="n">grad_values</span><span class="w">
      </span><span class="p">}</span><span class="w">
      </span><span class="n">x</span><span class="w">
    </span><span class="p">}</span><span class="w">
    
    
    </span><span class="n">original_shape</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">dim</span><span class="p">(</span><span class="n">image</span><span class="p">)[</span><span class="o">-</span><span class="nf">c</span><span class="p">(</span><span class="m">1</span><span class="p">,</span><span class="w"> </span><span class="m">4</span><span class="p">)]</span><span class="w">
    </span><span class="n">successive_shapes</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">list</span><span class="p">(</span><span class="n">original_shape</span><span class="p">)</span><span class="w">
    
    </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">i</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="m">1</span><span class="o">:</span><span class="n">settings</span><span class="o">$</span><span class="n">hyperparams</span><span class="o">$</span><span class="n">num_octave</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
      </span><span class="n">successive_shapes</span><span class="p">[[</span><span class="n">i</span><span class="m">+1</span><span class="p">]]</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="nf">as.integer</span><span class="p">(</span><span class="n">original_shape</span><span class="o">/</span><span class="n">settings</span><span class="o">$</span><span class="n">hyperparams</span><span class="o">$</span><span class="n">octave_scale</span><span class="o">^</span><span class="n">i</span><span class="p">)</span><span class="w">
    </span><span class="p">}</span><span class="w">
    </span><span class="n">successive_shapes</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">rev</span><span class="p">(</span><span class="n">successive_shapes</span><span class="p">)</span><span class="w">
    
    </span><span class="n">original_image</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">image</span><span class="w">
    </span><span class="n">shrunk_original_img</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">image_array_resize</span><span class="p">(</span><span class="w">
      </span><span class="n">image</span><span class="p">,</span><span class="w"> </span><span class="n">successive_shapes</span><span class="p">[[</span><span class="m">1</span><span class="p">]][</span><span class="m">1</span><span class="p">],</span><span class="w"> </span><span class="n">successive_shapes</span><span class="p">[[</span><span class="m">1</span><span class="p">]][</span><span class="m">2</span><span class="p">]</span><span class="w">
    </span><span class="p">)</span><span class="w">
    
    </span><span class="n">shpnum</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="m">0</span><span class="w"> </span><span class="c1"># for debugging</span><span class="w">
    </span><span class="k">for</span><span class="w"> </span><span class="p">(</span><span class="n">shp</span><span class="w"> </span><span class="k">in</span><span class="w"> </span><span class="n">successive_shapes</span><span class="p">)</span><span class="w"> </span><span class="p">{</span><span class="w">
      </span><span class="n">shpnum</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">shpnum</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="m">1</span><span class="w"> </span><span class="c1"># for debugging</span><span class="w">
      
      </span><span class="n">image</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">image_array_resize</span><span class="p">(</span><span class="n">image</span><span class="p">,</span><span class="w"> </span><span class="n">shp</span><span class="p">[</span><span class="m">1</span><span class="p">],</span><span class="w"> </span><span class="n">shp</span><span class="p">[</span><span class="m">2</span><span class="p">])</span><span class="w">
      </span><span class="n">print</span><span class="p">(</span><span class="n">paste0</span><span class="p">(</span><span class="s1">'running octave '</span><span class="p">,</span><span class="w"> </span><span class="n">shpnum</span><span class="p">))</span><span class="c1"># for debugging</span><span class="w">
      </span><span class="n">image</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">gradient_ascent</span><span class="p">(</span><span class="n">image</span><span class="p">,</span><span class="w"> </span><span class="n">settings</span><span class="o">$</span><span class="n">hyperparams</span><span class="o">$</span><span class="n">iterations</span><span class="p">,</span><span class="w"> </span><span class="n">settings</span><span class="o">$</span><span class="n">hyperparams</span><span class="o">$</span><span class="n">`step`</span><span class="p">,</span><span class="w"> 
                               </span><span class="n">settings</span><span class="o">$</span><span class="n">hyperparams</span><span class="o">$</span><span class="n">max_loss</span><span class="p">)</span><span class="w">
      </span><span class="n">print</span><span class="p">(</span><span class="n">paste0</span><span class="p">(</span><span class="s1">'finished octave '</span><span class="p">,</span><span class="w"> </span><span class="n">shpnum</span><span class="p">))</span><span class="c1"># for debugging</span><span class="w">
      </span><span class="n">upscaled_shrunk_original_img</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">image_array_resize</span><span class="p">(</span><span class="n">shrunk_original_img</span><span class="p">,</span><span class="w"> </span><span class="n">shp</span><span class="p">[</span><span class="m">1</span><span class="p">],</span><span class="w"> </span><span class="n">shp</span><span class="p">[</span><span class="m">2</span><span class="p">])</span><span class="w">
      </span><span class="n">same_size_original</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">image_array_resize</span><span class="p">(</span><span class="n">original_image</span><span class="p">,</span><span class="w"> </span><span class="n">shp</span><span class="p">[</span><span class="m">1</span><span class="p">],</span><span class="w"> </span><span class="n">shp</span><span class="p">[</span><span class="m">2</span><span class="p">])</span><span class="w">
      </span><span class="n">lost_detail</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">same_size_original</span><span class="w"> </span><span class="o">-</span><span class="w"> </span><span class="n">upscaled_shrunk_original_img</span><span class="w">
      
      </span><span class="n">image</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">image</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="n">lost_detail</span><span class="w">
      </span><span class="n">shrunk_original_img</span><span class="w"> </span><span class="o"><-</span><span class="w"> </span><span class="n">image_array_resize</span><span class="p">(</span><span class="n">original_image</span><span class="p">,</span><span class="w"> </span><span class="n">shp</span><span class="p">[</span><span class="m">1</span><span class="p">],</span><span class="w"> </span><span class="n">shp</span><span class="p">[</span><span class="m">2</span><span class="p">])</span><span class="w">
    </span><span class="p">}</span><span class="w">
    
    
    </span><span class="n">image_path</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
      </span><span class="n">gsub</span><span class="p">(</span><span class="s1">'/orig/'</span><span class="p">,</span><span class="w"> </span><span class="s1">'/dream/'</span><span class="p">,</span><span class="w"> </span><span class="n">.</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w"> 
      </span><span class="n">gsub</span><span class="p">(</span><span class="s1">'.jpg'</span><span class="p">,</span><span class="w"> </span><span class="n">paste0</span><span class="p">(</span><span class="s1">'_dream'</span><span class="p">,</span><span class="w"> </span><span class="n">setting_counter</span><span class="p">,</span><span class="w"> </span><span class="s1">'.png'</span><span class="p">),</span><span class="w"> </span><span class="n">.</span><span class="p">)</span><span class="w"> </span><span class="o">%>%</span><span class="w">
      </span><span class="n">png</span><span class="p">(</span><span class="n">filename</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">.</span><span class="p">)</span><span class="w">
    </span><span class="n">plot</span><span class="p">(</span><span class="n">as.raster</span><span class="p">(</span><span class="n">deprocess_image</span><span class="p">(</span><span class="n">image</span><span class="p">)))</span><span class="w">
    </span><span class="n">dev.off</span><span class="p">()</span><span class="w">
    </span><span class="n">print</span><span class="p">(</span><span class="n">paste0</span><span class="p">(</span><span class="s1">'finished '</span><span class="p">,</span><span class="w"> </span><span class="n">image_path</span><span class="p">,</span><span class="w"> </span><span class="s1">', setting # '</span><span class="p">,</span><span class="w">  </span><span class="n">setting_counter</span><span class="p">))</span><span class="w">
  </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span>

To leave a comment for the author, please follow the link and comment on their blog: Dan Garmat's Blog -- R.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)