Tag Archives: Neural Networks

Stepping Back to Race Forward

Deep Learning and Neural Networks: two words that have dominated the press and social media in the localization industry for the last couple of weeks. Google Research’s blog post last week about their success with neural networks to power a production scale machine translation engine sparked a lot of conversation.

I’ve been interested in neural networks for the last couple of years, researching what they’re good at and thinking of potential use cases within Vistatec. I’m not one for wading into debates particularly when I don’t have first hand experience to substantiate my view or add any unique insights. I will say that I’m very excited about this development though. It reinforces again that you cannot stay still in the technology business. New paradigms will shake the ground beneath you.

One of the aspects of NMT that intrigued me was how the encoding and decoding of variable length sentences was achieved given that neural networks essentially work with vectors. It turns out Word Embedding‘s (or Word Vectors) play a part. [Dear readers, if you fully understand the creation and implementation of these, please leave me a comment or reference.] Now I get semi-obsessed when I think I haven’t fully understood a concept and so ensued my journey of the last week:

Binge-watching Stanford’s CS224D and Coursera’s Machine Learning course; revising my secondary school and university calculus; and reading everything I could find on logistic regression, backpropogation and neural network fundamentals including Neural Networks and Machine Learning and Best Machine Learning Resources for Getting Started.

Having filled up my head with concepts and a rough mental map of how it all goes together, the next step was to play with a framework and get my hands dirty. But which one? It seems the ML Framework du jour is Google’s TensorFlow. So, sleeves rolled up, diet coke and nibbles, we’re oo… Linux! I have to have Linux?!

OK, I knew I’d have to assimilate Python but what ensued was another intravenous intake of not unknown but unfamiliar tasks. Provisioning a basic Linux box on AWS and remoting into it from Windows using Putty so I could install the Nvidia TensorFlow Tutorial. Install Docker. Learn the basics of Docker. Install a Linux GUI and figure out how to remote into that from Windows by configuring Remote Desktop protocols. Install Python and TensorFlow and … I have to stop to attend to other commitments.

So, like all great weekly television series this project will have to be continued with another exciting installment.

 

Variety is the Spice of Life

January has been wonderfully varied on the work front. Most days have brought new learning.

In thinking about my R&D Manifesto I decided it was time to revisit Neural Networks and Semantics. I’m not adverse to learning new development languages and environments but when you want to evaluate ideas quickly one will tend towards the familiar. For this initial reason Encog looks interesting.

The Centre for Global Intelligent Content, which VistaTEC have been industry partners of since its inception, received funding for a further two and a half years in October of last year. As a consequence there have been numerous meetings. It’s really exciting to see how the centre has evolved and honed its process for bringing innovation to the market. A key element of this process is the d.lab under the direction of Steve Gotz. In my view Steve has been one of the notable personalities in CNGL. He has a great broad knowledge of the technology, innovation and start-up landscapes and excellent business acumen. Two interesting pieces of technology were shown to centre members recently. The first named MTMPrime is a component which in real-time can assess translation memory matches along side of machine translation output and based on confidences recommend which one to use. The second is a machine translation incremental learning component which can profile a document and suggest the most efficient path to translating it given the algorithm’s analysis of the incremental benefit that would be realized from translating segments in a particular order. Basically it works out the bang-for-buck for translating segments.

In discussing semantics and disambiguation Steve pointed me at Open Calais. This is a service which like Enrycher parses content and automatically adds semantic metadata for named entities and subjects that it “recognizes”. The picture below shows the result of assign this post through the Open Calais Viewer.

open_calais

We’ve had some very interesting customer inquiries too. Too early to talk about them but I hope that we get more requests for these types of engagements and services. If any come to fruition I’ll blog about them later.

Finally, we did some small updates to Ocelot:

  • New configuration file for plug-ins,
  • Native launch experience for Windows and Mac, and
  • Native hot-key experiences within Windows and Mac interfaces.

Long may this variety continue.