Wow, June already. Time flies in the enjoyable world of translation and technology.
I embraced the cloud 6 years ago having evaluated the benefits of Platform and Software as a Service and believed in, what was then, a future vision of all kinds of intelligent distributed services which would be impossible to achieve with a private, internal infrastructure. It was interesting to see that light bulb flash on for non-cloud using attendees at Microsoft’s Red Shirt Dublin event with Scott Guthrie last week.
Scott took us on a whistle-stop tour of Azure facilities from Functions (a few lines of code executing logic on demand) to arrays of GPU’s running Deep Learning algorithms capable of doing face recognition and sentiment analysis.
Within the development team at work our utilization of such technologies continues: Neural Network Machine Translation; Adaptive Machine Translation; Continuous Integration; Distributed Services; and Serverless functions and logic.
At the Research end of the scale, having successfully completed our most recent European Project, I’ve been re-engaging with local research centers and interest groups. This month’s and last month’s Machine Learning Meetups were testament to how dominant Deep Learning is in driving business success and competitiveness.
And because working hard has to be balanced by playing hard I’ve ramped up sailing to three times a week.
The Cork 1720’s I go out in are just wonderful boats.
We started the year with some operationally complex, significant impact projects. Progress has been slower than I would have liked but ensuring we have a solid base upon which to build is critical to the overall success. My impatience is to realize some of the potential gains now but the collateral risk is too high. So, at the midpoint we are looking at a busy next two quarters to get everything we want done but the team is well capable.
Over the Christmas break I started to reflect on the nature of service provision in the Language Services industry in the light of new technologies coming out of machine learning and artificial intelligence advances and my own predictions of the influences upon the industry and the industry’s response to them.
There are the recent announcements of adaptive and neural network machine translation; pervasive cloud platforms with ubiquitous connectivity and cognitive capabilities; an upsurge in low-cost, high-benefit open source tooling and frameworks; and many mature api’s and standards.
All of these sophisticated opportunities really do mean that as a company providing services you have to be informed, adaptable, and agile; employ clever, enthusiastic people; and derive joy and satisfaction from harnessing disruptive influences to the benefit of yourselves and your customers.
I do have concerns: How do we sustain the level of investment necessary to stay abreast of all these influences and produce novel services and solutions from them in an environment of very small margins and low tolerance to increased or additional costs?
Don’t get me wrong though. Having spent the last 10 years engaging with world-class research centers such as ADAPT, working alongside thought leading academics and institutions such as DFKI and InfAI, participating in European level Innovation Actions and Projects, and generally ensuring that our company has the required awareness, understanding and expertise, I continue to be positive and enthusiastic in my approach to these challenges.
I am satisfied that we are active in all of the spaces that industry analysts see as being currently significant. To whit: ongoing evaluations of adaptive translation environments and NMT, agile platforms powered by distributed services and serverless architectures, Deep Content (semantic enrichment and NLP), and Review Sentinel (machine learning and text classification).
Less I sound complacent, we have much more in the pipeline and my talented and knowledgeable colleagues are excited for the future.
Deep Learning and Neural Networks: two words that have dominated the press and social media in the localization industry for the last couple of weeks. Google Research’s blog post last week about their success with neural networks to power a production scale machine translation engine sparked a lot of conversation.
I’ve been interested in neural networks for the last couple of years, researching what they’re good at and thinking of potential use cases within Vistatec. I’m not one for wading into debates particularly when I don’t have first hand experience to substantiate my view or add any unique insights. I will say that I’m very excited about this development though. It reinforces again that you cannot stay still in the technology business. New paradigms will shake the ground beneath you.
One of the aspects of NMT that intrigued me was how the encoding and decoding of variable length sentences was achieved given that neural networks essentially work with vectors. It turns out Word Embedding‘s (or Word Vectors) play a part. [Dear readers, if you fully understand the creation and implementation of these, please leave me a comment or reference.] Now I get semi-obsessed when I think I haven’t fully understood a concept and so ensued my journey of the last week:
Binge-watching Stanford’s CS224D and Coursera’s Machine Learning course; revising my secondary school and university calculus; and reading everything I could find on logistic regression, backpropogation and neural network fundamentals including Neural Networks and Machine Learning and Best Machine Learning Resources for Getting Started.
Having filled up my head with concepts and a rough mental map of how it all goes together, the next step was to play with a framework and get my hands dirty. But which one? It seems the ML Framework du jour is Google’s TensorFlow. So, sleeves rolled up, diet coke and nibbles, we’re oo… Linux! I have to have Linux?!
OK, I knew I’d have to assimilate Python but what ensued was another intravenous intake of not unknown but unfamiliar tasks. Provisioning a basic Linux box on AWS and remoting into it from Windows using Putty so I could install the Nvidia TensorFlow Tutorial. Install Docker. Learn the basics of Docker. Install a Linux GUI and figure out how to remote into that from Windows by configuring Remote Desktop protocols. Install Python and TensorFlow and … I have to stop to attend to other commitments.
So, like all great weekly television series this project will have to be continued with another exciting installment.
I’m using this post to both keep my blog live and also organise my own thoughts on everything that’s been going on over the last six weeks.
We deployed the distributed components of our XTM integration to production and have pushed a number of projects through it. We delivered this project through a series of backlog driven sprints. It wasn’t without its challenges: requirements changes, unforeseen constraints and aggressive timelines. Some elements of test-driven development were employed (and very useful) as were domain-driven design.
On Wednesday I was delighted to receive a tweet from @kevin_irl announcing the open source publication of their .NET XLIFF 2.0 Object Model on GitHub. Coincidentally Wednesday was also the day that my copy of Bryan Schnabel’s “A Practical Guide to XLIFF 2.0” arrived from Amazon. One of my developers, Marta Borriello, is currently working on a prototype of the XLIFF Change Tracking module which includes support for inline markup inside of Ocelot with the hope that this will be part of XLIFF 2.1.
Machine Learning is one of my primary interests but sadly a tertiary focus after the “day job” and family (don’t you listen to their denials). Hot on the heels of machine learning is programming paradigms and languages. So with a peak in travel I decided to combine both and downloaded “Machine Learning Projects for .NET Developers” by Mathias Brandewinder and four F# courses from Pluralsight and got myself up to speed on functional programming. This turned out to be a really valuable exercise because I got to understand that functional programming gives you much more than immutable data and functions as first class objects. There’s sequences, partial application and composition to name a few. One day I plan to re-implement Review Sentinel’s core algorithms in F# but don’t hold your breath for a post about that.
I’m just back from a FREME Project hackathon in hyp-zig. Two days of enthusiasm-fuelled fun creating and querying linked data with guys at the forefront of of this exciting technology. We hacked topic/domain detection/identification using the services of the FREME platform.
FREME Team: Francisc Tar, Felix Sasaki, me, Jan Nehring, Martin Brümmer, Milan Dojchinovski and Papoutsis Georgios.
Today I attended an ADAPT Governance Board meeting. ADAPT is the evolution of the CNGL CSET into a Research Centre. I think ADAPT has a great research agenda and has numerous world-class domain thought leaders. I’m looking forward to working with the ADAPT team during 2016 to push a few envelopes. My engagement with academia and research bodies at both national and European level over the last 10 years has been of great tangible and intangible value (no I don’t just mean drinking buddies). I have to thank Peter Reynolds (who will never let me forget it), Fred Hollowood (who will have an opinion about it) and Dave Lewis (who will be typically English and modest about it) for helping me overcome the initial inertia and Felix Sasaki who has made the bulk of it demanding, rewarding and enjoyable.
There, that’s better. Neuron activity stabilized and synapses clear.
I try to maximize the dark evenings for evaluating tools and researching topics that I don’t get time for during a normal working day.
First up for the New Year is LanguageTool. Automated Quality Assurance is an important aspect of many production processes and at VistaTEC we have deployed various commercial and in-house tools. My motivation for looking at LanguageTool was its use of Part of Speech tagging and user definable rules which can be combined with regular expressions to encode sophisticated linguistic checks.
My test domain was Marketing translations. Content full of emotive, symbolic and suggestive language.
The custom rule encoding is necessarily verbose given that the serialisation format is XML. Our internal tool, Cerberus, suffers from the same characteristics – elements and escaping of regular expression meta characters. Some of the advanced rule constructs are initially difficult to grasp. We ran many small tests in order to get to understand the operation of functionality like skip scope. Hopefully the evolving Rule Editor will support more advanced rule constructs soon and this will aid learning and perhaps speed up rule writing.
We were not able to build rules for all of the constructs that we wanted. Extending the tool via Java for these is a possibility. That will be another days work.
Next up has been Neural Networks. I’ve been viewing Andrew Ng’s Machine Learning lectures on Coursera. Learning is always helped by consulting several references and James McCaffrey’s articles have been straight-forward to understand. I really like that James included worked examples in his articles – it’s a great way of being able to check your own understanding (or lack of). Finally, it’s nice that the code examples are in C# rather than the ubiquitous Python (I can read it but let’s say it’s not my native tongue).
I’d prefer to be resident in warmer climes between November and March so that I could be out more without the need for layers of thermal underwear but I do love the different seasons.
Train journeys are a great opportunity to experience a place from a different perspective and learn something new. (I find it difficult to engage in activities that don’t have a purpose.)
The initial part of the train journey from Dublin to Belfast follows the eastern coast of Ireland. There are some wonderful, long sandy beaches on this stretch.
As the train nears Drogheda it vears more inland so as to cross the River Boyne which separates county’s Louth and Meath. The segment between Drogheda and Dundalk is characterised by green farm land and large detached houses. Then as you near Newry there are more hills.
I decided I wanted to take a second run through of Andrew Ng’s Machine Learning course available free on Coursera. I have taken several of Coursera’s coursers (knowledge junkie) and I have to say that Andrew stands out as an awesome teacher. Despite the terminology-laden and mathematical nature of the course content, Andrew puts the concepts in context, explains them clearly and moves through the syllabus in a series of logical, developmental steps which are easy to assimilate.
Newry – my stop. Heading to Freddy-fest where no doubt the liberal consumption of alcohol will destroy precisely the brain cells that hold the recently refreshed knowledge of linear regression.