Search Engines, Platforms and the Past and Future of A Personally-mapped Internet

01 Jul 2020

14-minute read

Note: This was written in 2019 in response to a call for writing but not previously published. I have added a single paragraph about linktree and noted its addition, done some minor typo-correcting and published here now.

Search As Secret Weapon

In the summer of 2000, I interned for Philadelphia Weekly, at the time a weekly print publication. A significant part of my internship was fact-checking articles. When I arrived, the new person, I had a secret super skill: Google. Before the advent of Google, there were dozens of web search engines and most users tried each of them to see which would produce the best results for their search. Whereas fact checkers at the newspaper previously relied on phone calls and footwork, I had the additional ability to quickly look for answers across the web because Google, one of a large number of search engines at the time, was just so much better and actually could find useful results, but most folks did not yet know of it, or assumed it was no better than the other search engines out there.

Prior to Google, searching could be a difficult affair with sites returning more general links, few results, or seemingly random miscellany. In that era there was AskJeeves, with its cartoon butler logo; HotBot - which never found useful results for me; Yahoo!; MSN; and AOL. They all attempted to be a comprehensive platform before the widespread use of that term, by offering search, mail, messenger and other features. Even before the era of the web browser, when the Internet consisted of using the shell to connect to text-based bulletin boards there was GOPHER, a pre-WWW protocol for distributing and searching for text on the internet, using its Archie and Veronica search engines.

Archie Search
Archie Search in the browser

Some search engines specialized in particular topics, or one would have to narrow down a search by specifying particular categories to search within, selecting narrower and narrower categories or refining one’s search. Hopping between half a dozen search engines until you found what you were looking for could be a lengthy process. Google, for me and many others, felt like the first one stop search engine that just worked.

As word got out that Google was better at search than its competitors it began to become many folks’ homepage, and link aggregator sites like BoingBoing and Reddit, which described itself as the ‘front page of the internet’, flourished. New users were streaming onto the net, and they were looking for content. Unlike early adopters, they were less invested in participation and more inclined toward spectating. This was the perfect opportunity for a less finicky, more streamlined, centralized version of the internet. This was the beginning of the rise of the social media behemoths.

Sharing in the pre-platform days: That Humble Scrap Heap, The Links Page

Prior to the wide spread of these platforms with closed ecosystems (MySpace, Facebook, Google…) building a website required basic knowledge of HTML, the language that underlies the web, and some CSS for style. Alternatively, some CD-Roms contained basic web-page making tools using the What-You-See-Is-What-You-Get (WYSIWYG) approach to design, a one-size fits all approach before there was a need for marking up a website’s appearance for desktop, tablet and phone. Rather than posting thoughts, notes, research or personal news to a social media site, one posted to a personal website, maybe on the home landing page, or on a separate linked blog or news page. Then, as now, personal websites contained a Home Page and usually a Contact page. Then there may have been any number of other pages for specific topics. One thing that was common: the links page.

If you read the Wikipedia’s Links page you might get the idea that they were of little value. Only four sentences long, the main indicator of a links page is that they contained a list of links the web page owner found notable to mention (emphasis mine).

Prior to widespread and well-functioning search engines, links pages were how you could find out about things you were interested in. Or, more accurately, it was where other folks listed things they were interested in. When browsing someone’s website, I would read their home page and news, and then inevitably browse their links page, which would whisk me off to other corners of the net, and this was a time before we had the ability to open multiple tabs, resulting in a constant forwards and backwards march in the browser.

If news sites were the town crier and chat rooms the crowded market stalls of the 90s internet, Link Pages were the alleyways or scrap heaps, a grab bag of more or less uncategorized mini portals to friends, obscure hobbies and idiosyncrasies.

They could be a short title, just the name of the website page, or they might contain a sentence describing what lay there. Link pages were static, a personal selection of one’s all-time favorite sites, or they were pruned or grew.

In that 1.0 era, Links pages made a lot of sense. Consider the size of the web at that time. There were significantly less web pages in the world, and it wasn’t always obvious where to go, or what the Internet was good for. One might still have those questions.

When I started using a web browser in middle school, somewhere in 1995-1996, there was a jump from 20,000 to 250,000 total web pages over the course of that year.

When my family first acquired a personal computer for our home I checked out a book at the library on how to use a web browser. Beside explaining the browser, toolbar and following links going forward and backward, a significant part of the book was a directory listing for interesting websites to visit, along with a screenshot and a blurb of what you’d find. Using this book, I could pick a topic of interest and type in the URL, hoping to find something interesting. Official government websites such as NASA or scientific research mingled in the pages with internet-connected Soda Machines and Coffee Makers and sites dedicated to the splinter religion Church of the Sub-Genius. By the end of 1999 there were a bit over 3 million websites. That number seems high, but by 2005 in the dawning of Facebook and Flickr, there were about 65 million websites, and of course that number exploded in the years afterward.

The World Wide Web
Not the book I read in 1995, but probably pretty similar!

Within a few years it became less essential to consult a book to find out about useful and interesting sites. Websites started to update more regularly, or to change their URL. Link rot, the process of previous URLs becoming defunct due to not paying bandwidth fees or site architecture changes or deletion became a regular occurrence. A new crop of websites dedicated to listing the latest interesting thing on the net started to appear. These sites had a single or several editors who would regularly post links out to updated or new content elsewhere on the net, usually with a short summary and maybe even a screenshot or image. These precursors were the early content curators.

Whole media empires and billions of dollars of income have been built on this technique of linking to content devised and hosted elsewhere. BuzzFeed, Vox, Gawker Media, and others exemplified the creation of click-bait, listicles, or articles consisting of little more than an appropriated photo, 1 paragraph explanation and link to the source. Boing Boing functions this way as well, with tiny writeups and links out to others’ content yet it was founded as a print zine (“The World’s Greatest Neurozine”), originally contributing to the cyberpunk subculture. It still covers largely technology, science fiction and politics.

These edited links as news sites might have used the same technology under the hood, but a personal link page functioned as personal bookmarks you wanted to show off or retain for use. One of the earliest sites serving as a proto-social-media was del.ici.ous, which in the early 2000s was a web-based bookmarking links site to save and share one’s own links. Artists were early adopters and internet Surf Clubs grew out of this nascent community. I didn’t participate so much in this culture at that time as it felt exclusionary and I wasn’t interested in the aesthetic of defaults and boredom on the Internet. Though I was a huge fan of fellow traveler excessive eccentrics like Borna Sammak’s Double Happiness which in retrospect might have been less of a group effort and more of a messy website-as-collage practice.

Although HTML is arguably much simpler to get started than languages like Java or Javascript, it was still a barrier to entry. The closed ecosystems gave a framework and made it easy to link to articles or pages elsewhere, to upload your own text as journal entries or any way you saw fit, and post photos. While not a comprehensive list of everything one might want to share online, this covers the vast majority of many users’ needs. Myspace allowed one to make a personal page and there were a number of free web-based custom page builders that let you choose colors, fonts, images, and page structures - spitting out HTML/CSS code you could inject into your personal Myspace page.

With the rise of personal pages on platformed sites like Myspace one’s own page combined a short bio, descriptions or testimonials. Users browsed galleries of people, and later images, instead of links to other content or news. Then the ability to post pithy statements and pose questions arose and soon linking to articles, sites. The need for an outside website, much less a personal links page was considered moot.

While personal links pages dwindled personal bookmarking seems to be as strong as ever, this time living exclusively on toolbars and browser menus. Specialized bookmarking, for images and hobbies (Pinterest), Articles (Pocket), or design (Are.na) favor a public list-making or streaming of one’s selections and are web-based corporate services, with free and premium tiers in a closed ecosystem without a way to download or migrate one’s data, a constantly-updated un-archived personal directory.

(Note: This paragraph was written over a year after the other text in this post) Ironically, at the moment I post this a very-specific freemium links-page tool is thriving in one corner of the web: Linktr.ee is a simple tool designed to make a single URL link page. Read its description (“Connect audiences to all of your content with just one link”) which is a sort of corporatized co-optation of links pages into a branding source for one’s own now-distributed content across a range of platforms/URLs. Instagram users in particular are using the tool to direct their fans to multiple sources: one’s own personal collection of links on other platforms (soundcloud, twitter) or as a source to distribute resources or activist causes for Black Lives Matter.

Prior to the era of personal branding, the links page served in quite an opposite fashion. Brandon Avery Joyce of The Universal Research Group writes in his blog The Enthusiast about the humble links page as a decentralized technology. He celebrates their power to support a form of personal empowerment in the face of centralized platforms.

Bookmarks and links pages are an alternative to the feeds and timelines of social media, meant to supplement rather than supplant them. The very attributes that have made social media so successful, as far as connectivity, have also given rise to its considerable failings….Links pages, which have fallen into only nostalgic use, encourage more self-direction and decentralization, and shape the overall structure of the internet slightly more conscientiously (since search engines are greatly though not entirely organized by these links, in ways that social media stifle). Even the random or inattentive clicking across the news links helps break up the overconsolidations and predeterminations of the feed, done not as some revolutionary act, but merely to encourage better habits.–Joyce

In this perspective, links pages are the Google/Facebook/Amazon/Microsoft alternative, holding the power to construct a personal structure of the web, a kind of pre- and post-figurative internet. Joyce ends with an invitation. “I encourage you to bookmark this links page, or create your own, rather than always and only waiting for news and articles to float down the stream of social media.”

Back To The Future

The past several years has seen a developing community of creators and early adopters of the decentralized web, including the DAT network, decentralized apps, and the Beaker Browser. Beaker Browser is an experimental browser for the decentralized web, and rather than hosted sites on corporate sites such as Amazon’s AWS (Amazon Web Services, which hosts a significant part of major web platforms), the DAT network instead parcels out the bandwidth of a website, simultaneously serving the HTML page for a site on the site creator’s machine and any viewer that wishes to view and agree to co-host the site. In this way, the decentralized web functions more akin to Napster and the torrent community where the underlying site architecture can be divided up and served and passed between the community, eliminating centralized gatekeepers.

While there is a thriving early community building and using the decentralized web, most decentralized sites’ name and web address are a significantly long string of alphanumeric characters, and there is currently no specific search engine for DAT websites. For this reason, the default browser homepage is effectively a links page, and a number of pages with suggested sites to browse are published by those building sites on the decentralized web. These pages are akin to the early links pages - helping to grow content, and to build community in the early days before a (possible) page explosion, when browsing is more akin to a stroll.

A Tiny Call For Linking

There is a different way to be online. It’s not about archeology as such but about the difference between creating one’s own (web) home versus the significantly more constrained world of a closed platform ecosystem. It’s also the difference between creating one’s own directed intentional portal or hallway of doors to elsewhere, and the reactive always-in-flux churn of one’s social media stream.

The links page is a reminder of the un-curated and disorganized, grouped around that which is found elsewhere. It shoots you out of the links page to a series of new destinations. In contrast, the platform personal page and stream blurs links, writing and attention-span grabbing advertisements mingling in the same column as our health updates, party photos or news headlines. The links page can let us consume without being con-sumed. We can still waste our time but intentionally so. This is a way to un-advertise yourself - to separate your identity from some giant corporate machine that sucks up your profile data and sells it. A personal links page is a way to de-instrumentalize yourself and remove the likes.

As a new generation of artists, programmers, and many more shape this Web, what lessons can be learned from the early web? Three decades have passed and the same basic tool of web creation is available: HTML, the language of the web, is still waiting for use and abuse. With a minimum of basic training, one can craft a simple link page, for personal use as a home page, to take a break from the constant churn of the stream, and as a location for sharing and building community. Importantly, those that build the web and have an interest in influencing its future can contribute with the creation of tools and supporting initiatives informed by the unpretentious, ordinary Links page. How might this guide and inform our work?

The link page can serve as a source of calm, of reassurance, and of friendship. It can remain in flux, or hold steady. It can be shared or hidden in a secret webpage. It’s there to be used, a minimal, flexible tool one can shape and grow.