07 December 2013

Taking It On The Road, Travel Technology 2013

I spent much of this year mountaineering in Europe, and re-visiting India after 34 years. Since my father's passing earlier this year, I've been free to travel again. My father was my chief technology influence. People often ask how I got into technology, since my education was in psychology and most of my career was in tourism. It was all due to my father, Lucian J. Endicott Jr.,who worked nearly three decades for IBM, and then became a professor of computer science before retiring altogether.

On this journey I've been watching closely to see what technologies I find most useful. Unlike most of the young people traveling today, I'm not traveling with a phone. I did have an Apple iPod Touch for awhile, which I enjoyed, but passed it on in favor of the new Google Nexus Android tablet. I find phones and tablets great for everything other than real work, like programming. I did buy the most economical, top rated Consumer Reports laptop for students, and have been very happy with it.

A man can only travel with so many devices though; so, the Apple iTouch and Android tablet both went to nieces, and I'm still happily traveling with my affordable laptop. In both Europe and Asia I have found locally available, prepaid "data cards" or "dongles", basically a phone chip on a USB stick, very helpful for freeing myself from dependence on wifi. However, some of the new, higher end phones come with built-in wifi "hotspot" capability, which I've seen quite a few young people using with their laptops. Without a phone per se, Skype has proven super convenient, especially premium Skype to local landlines and SMS, for literally calling from anywhere to anywhere.

I have to say that I use Skyscanner a lot, and feel it's saved me a tremendous amount of money. The only caveat is that some of the smallest new budget airlines are not included. For accommodation, I have tried both Couchsurfing and Airbnb for the first time this trip. I've actually found Couchsurfing more useful for meeting interesting, colorful people at my destinations than for easily organizing free overnights. I have also been surprised by the amount of people running accommodation operations under the radar via Airbnb, rather than truly private persons renting spare rooms, though I've been satisfied with the service.

Problematic as it may be, I do find myself using Wikitravel precisely because it provides less information rather than more. I like Wikitravel because it gives me a really quick overview of the high points, what to see and do, even for out of the way destinations. I sometimes download the entire page for offline reading, when there is no wifi available. I find both its strongest and weakest points are accommodation. Strong because anyone can add to it, so often gets places under the radar, and weak because it's totally unorganized and un-rated. Because of this deficiency, I find myself often referring to TripAdvisor reviews to double check the lower end accommodations.

Another relatively new technology I'm using a lot is Google Maps, of course frequently for directions, but also particularly for the "Search nearby" capability. I find Google Maps Search nearby capability delightful for discovering new places of all kinds, many perhaps never visited by tourists before. Especially in India directions should be taken with a grain of salt, because places usually seem to be "pinned" imprecisely, so caveat emptor. I find screenshots great for capturing Google Maps directions, easily cut and paste with the "prt sc" key for convenient offline reading.

More than ever before, travel for me is more about people than places. These days I love to visit with friends, old and new, at home or abroad. The reality is that people are now using Facebook more for personal communications than email. Facebook even makes it easier for people traveling in the same regions to meet up along the way. Another reality is that places, particularly low end places, are as likely to have Facebook pages as websites, essentially turning Facebook into its own parallel universe – no other Internet required. In fact, I only use Facebook for people, places/pages and events – and no other bells or whistles, such as innumerable travel apps, etc.

It was Facebook (and my nieces) that finally made me break down and get a small camera (Nikon Coolpix) for posting travel snaps as I go along, often the same day. Though that could be the number one use for phones I'm seeing on the road, not only for taking pictures but also for uploading them in virtual realtime….

04 September 2013

Dissecting the Summarization Process

This is in effect a mid-2013 progress update. As with many of my blog posts, this is as much a status update for me to get a better handle on where I'm at as it is to broadcast my progress.

mendicott.com is a blog reflecting on my journey with the overall project. This blog started seven years ago, in 2006, with my inquiry into The difference between a web page and a blog.... I had then returned from something like five years of world travel to find the digerati fawning over the blogosphere. At first, I failed to see the difference between a blog and a content management system (CMS) for stock standard web pages. Upon closer examination, I began to realize that the real difference lay in the XML syndication of blog feeds into the real-time web.

meta-guide.com is an attempt to blueprint, or tutorialize, the process. My original Meta Guide 1.0 development in ASP attempted to create automated, or robotic, web pages based on XML feeds from the real-time web. Meta Guide 2.0 development was based on similar feed bots, or Twitter bots, in an attempt to automate, or at least semi-automate, the rapid development of large knowledgebases from social media via knowledge silos. Basically, I use knowledge templates to automatically create the knowledge silos, or large knowledgebases. The knowledge templates are based on my own, proprietary "taxonomies", or more precisely faceted classifications, painstakingly developed over many years.

gaiapassage.com aims to be an automated, or semi-automated, summarization of the knowledge aggregated from social media by feed bots via the proprietary faceted classifications, or knowledge templates. Right now, I'm doing a semi-automated summarization process with Gaia Passage, which consists of automated research in the form of knowledge silos being "massaged" in different ways, but ultimately manually writing the summarization in natural language. This is allowing me to analyze and attempt to dissect the processes involved in order to gradually prototype automation. Summarization technologies, and in particular summarization APIs, are still in their infancy. Examples of currently available summarization technologies include automatedinsights.com and narrativescience.com. The overall field is often referred to as automatic summarization.

In the future, the Gaia Passage human readable summarizations will need to be converted into machine readable dialog system knowledgebase format. The dialog system is basically a chatbot, or conversational user interface (CUI) into a specialized database, called a knowledgebase. Most, common chatbot knowledgebases are based on, or compatible with, XML, such as AIML for example. Voice technologies, both output and input, are generally an additional layer on top of the text based dialog system.

The two main bottlenecks I've come up against are what I like to call artificial intelligence middleware, or frameworks, the "glue" to integrate the various processes, as well as adequate dialog system tools, in particular chatbot knowledgebase tools with both "frontend" and "backend" APIs (application programming interface), in other words a dialog system API on the frontend with a backend API into the knowledgebase for dynamic modification. My favorite cloud based "middleware" is Yahoo! Pipes, which is generally referred to as a mashup platform (aka mashup enabler) for feed based data; however, there are severe performance issues with Yahoo! Pipes -- so, I don't really consider it to be a production ready tool. Like Yahoo! Pipes, my ideal visual, cloud based AI middleware could or should be language agnostic -- eliminating the need to decide on a single programming language for a project. I have also looked into scientific computing packages, such as LabVIEW, Mathematica, and MATLAB, for use as potential AI middleware. Additionally, there are a variety of both natural language and intelligent agent frameworks available. Business oriented cloud based integration, including visual cloud based middleware, is often referred to as iPaaS (integration Platform as a Service), integration PaaS or "Integration as a Service".

The recent closure of the previously open Twitter API with OAuth has set my feed bot, or "smart feed", development back by years. Right now, I'm stuck trying to figure out the best way to use the new Twitter OAuth with Yahoo! Pipes, for instance via YQL, if at all. And if that were not enough, the affordable and user-friendly dialog system API, verbotsonline.com, that I was using went out of business. There are a number of dialog system API alternatives, even cloud based dialog systems, but they are neither free nor cheap, especially for significant throughput volumes. Still to do: 1) complete the Gaia Passage summarizations, 2) make Twitter OAuth work, use a commercial third party data source (such as datasift.com, gnip.com or topsy.com), or abandon Twitter as a primary source (for instance concentrate on other social media APIs instead, such as Facebook), 3) continue the search for a new and better dialog system API provider.

Most basically, the Gaia Passage project is a network of robots that will not only monitor social media buzz about both the environment and tourism but also interpret the inter-relations, cause and effects, between environment and tourism -- such as how climate change effects the tourism industry both negatively or positively, or even what effects the weather has on crime trends for a particular destination -- as well as querying these interpreted inter-relations, or "conclusions", via natural language. If this can be accomplished with any degree of satisfaction, either fully automated or semi-automated, then the system could just as easily be applied to any other vertical. Proposals from potential sponsors, investors, or technology partners are welcomed, and may be sent to mendicot [at] yahoo.com.

13 March 2013

A New Website For A New Age: GaiaPassage.com

GaiaPassage.com is subtitled "Marcus L Endicott's favorite tips for green travel around the world".  I'm calling it a deep green, eco-centric travel guide to the whole Earth.  My Gaia Passage project will be a handwritten ecotourism guide to the entire world, based on the circa 250x ccTLD.  The general idea is to write a "white paper" for every country in the world, on environmental and cultural conditions, issues, and who is doing what about them, as well as examining both how they affect tourism and how tourism affects those issues. Anyone could write a lot about something, but the idea here is to provide "snapshots", or "bite sized" summaries, of only the best information and contacts.  The name "Gaia Passage" originally came from my pre-Internet (mid-1980s) travel tips newsletter. The site is a work in progress; so far, I've completed the entire Western Hemisphere:
GaiaPassage.com is handwritten, but based on automated research and automated outline. Primary research is based on data mining 20 years of Green Travel archives. Secondary research is based on multiple years of Meta Guide Twitter bots archives. Significance is based on primary sources in the form of root website domains, and/or secondary sources in the form of Wikipedia entries. In other words, if there is not a root website domain name or a Wikipedia entry then it is unlikely to be included. (However, almost anything may be included in Wikipedia - if properly referenced.) 

I have noticed that many websites of smaller concerns are going down, offline, apparently due to the economic downturn. However, social media such as Twitter and Facebook do present affordable alternatives to owning a root domain website, and I will take these into consideration when appropriate. (In other words, when something is really cool.)  I have also noticed a lot of people using Weebly to make free websites. (Note, GaiaPassage.com currently uses the free Google Sites platform.)

In the early evolution of a website, especially large projects, it's important to first have the "containers" in place as "placeholders", which is no small task in itself. With circa 250x countries and territorial entities, that's a whole year's fulltime work for one man, revising one country per working day. This would mean initial completion by December 2013. Eventually, GaiaPassage.com entries may morph into socialbots, or conversational assistants, containing not only all the knowledge about sustainable tourism gleaned from past Green Travel archives, but also current knowledge resulting from the Meta Guide Twitter bots.

In my previous blog, 250 Conversational Twitter Bots for Travel & Tourism, I detailed my 250x Meta Guide Twitter bots, one for every country and territory in the Internet ccTLD.  Basically, I've spent the past five years working on artificial intelligence and conversational agents - and tweeting about it all the while (links below).  I had been using Twitter extensively as a framework; however, Twitter has become increasingly protectionistic, most dramatically illustrated by the high profile 2012 Twitter-LinkedIn divorce. The Twitter API has become a moving target, which is just too costly for me to keep playing catch up.  In short, I find the "Facebook complex" of Twitter management immensely annoying, and concluded to stop contributing original content; so, my New Year's resolution was to stop tweeting manually at least for all of 2013.  Further, my excellent dialog system API, VerbotsOnline.com, went out of business in 2012.  Any other good dialog system API I found to replace it turned out to be much too expensive.  As a result, all my conversational agents are shut down, at least for 2013.  My hope is that the sector will shake out and/or advance during the year, and better or at least more affordable conversational tools will become available next year.