Google is Killing Our Productivity. What We Can Do About It?

Sam Texas -

tldr; Information retrieval is a critical part of creative work. Google's once awesome search product is now a tool of distraction. In response I share some ideas about new ways of working with "workspace" search.

All work requires "looking it up"

I've been a developer, a manager, a cook, and an old Mercedes station wagon repair man.

At every turn in those careers (some shorter lived than others) I have used information retrieval as a part of my workflow:

  • In the kitchen I look up recipes.
  • I went onto forums, into old manuals, and specialist mechanics to figure out how things work on that old car.
  • Good old fashioned foolish "hold my beer" moments where I just had to take my best guess and hope for the best.
  • Ask my boss, mentor, or older folks that are around (and hope they know what they are talking about)

The Good Old Days

There was a time when Google shortened the path to answers. Once we learned how to plug in the right keywords the right way it would turn over some hidden stones and reveal the gems underneath. In short order we were back to work armed with new information that would become knowledge.

But those days are gone.

The shareholders and advertisers took over and monetized the hell out of us.

We were always the product.

Google shortens the distance between our eyes and advertising

Search something as simple as "Learn Django" and be greeted with this: a page full of ads and no organic results. Is this relevancy?

Advertising and SEO sewage masquerading as content

That field of relevant information is now a minefield of advertising and SEO sewage masquerading as content.

This is especially true for developers. And even more true for new developers.

It seems that I am frequently back on Google in search of a code snippet, a bug, or a docker container that already does the thing that I want.

The flow is always the same:

  1. Alt-tab to browser
  2. Open google
  3. Search something (this is a skill to develop)
  4. Scan past all the ads, sketchy SEO'd sites, and hunt for what might be the right link. (This is another skill unto itself, and honed after years of bad clicks)
  5. Click the link into new tab (there will be more new tabs as I hunt and peck)
  6. Eventually I find a few candidate pages that might point me to a good solution.
  7. Rinse, repeat forever.

By the time I find something worth trying I've broken the flow and restart the climb back up the productivity curve.

"OK, so I get it. Google no longer has your best interest at hear (if they every did). What do you propose?"

Glad you asked.

I want focused results (eg. Less is more)

In observing my own behavior I see there are a handful of resources I want to tap into when doing my work (development for example):

  • Documentation
  • Code snippets
  • Stack traces
  • Community / Forums
  • Libraries and Plugins

These come from only a few places. I don't need (or want) to walk the vast expanse of the web to only keep coming back to these same results. Google used to do a fine job of filtering and offering relevance, but again – those days are long gone.

I want a workspace that focuses my attention on these few resources. I can jump into documentation, code, or find help from the curated resources offered by a niche/vertical search engine.

Niche (aka vertical) Search is worth exploring

This is what I'm working on – a system of modules that stitch together as a focused, niche search engine. It is self-curated (by me or a community, or other entusiasts/subject matter experts) with the sole purpose to only return results optimized to my workflow.

I have:

  • Scripts to acquire content via Web, RSS, and APIs
  • A database to store, retreive, and sort the information to my needs.
  • Simplified and controlled interfaces optimized to how I want to work, and how I want to consumer the information.

The narrow scope of the project brings a few interesting side-effects:

  1. Shallow tech stack means I (or a small team) can understand all parts of it with relative competence.
  2. Narrow focus of content means I don't have scaling issues in terms of compute, network, or storage. Only a few gigs at most.
  3. Growing too large means that it is better to create a new engine with a narrower focus. This scales horizontally, but with some overhead on administration.
  4. We move the challenge from the hard problem and opaque solutions of AI to the clear and simple and marketable solutions of human Curation.
My niche/vertical search engine workspace.

So this is what I've been thinking about. Over the next few weeks, when working on Django projects, I will make this my first destination when seeking answers. I am curious to see if it actually improves my workflow, focus, and productivity.

A second hypothesis I will test is if this can work for others working in other ecosystems such as Javascript or Go language.