My favorite architectures are trained in the multilingual setting, where a single model translates from any source language to any target language, and in weakly supervised (or entirely unsupervised) settings. Machine Translation has usually been approaches with phrase-based statistical techniques, but neural networks are quickly becoming dominant. WaveNet) that produce raw audio signal outputs.
Speech synthesis has historically been approached with various stitching mechanisms, but today the state of the art models are large ConvNets (e.g. A very related, often cited humorous quote attributed to Fred Jelinek from 1985 reads “Every time I fire a linguist, the performance of our speech recognition system goes up”. Speech recognition used to involve a lot of preprocessing, gaussian mixture models and hidden markov models, but today consist almost entirely of neural net stuff. More recently, we don’t even trust ourselves to hand-code the architectures and we’ve begun searching over those as well. ImageNet) and searching in the space of Convolutional Neural Network architectures. Since then, we discovered much more powerful visual features by obtaining large datasets (e.g. Visual Recognition used to consist of engineered features with a bit of machine learning sprinkled on top at the end (e.g., an SVM).
#SPOTIFREE 2.0 CODE#
In each of these areas we’ve seen improvements over the last few years when we give up on trying to address a complex problem by writing explicit code and instead transition the code into the 2.0 stack.
Let’s briefly examine some concrete examples of this ongoing transition.
#SPOTIFREE 2.0 SOFTWARE#
Software (1.0) is eating the world, and now AI (Software 2.0) is eating software. Because of this and many other benefits of Software 2.0 programs that I will go into below, we are witnessing a massive transition across the industry where of a lot of 1.0 code is being ported into 2.0 code. It turns out that a large portion of real-world problems have the property that it is significantly easier to collect the data (or more generally, identify a desirable behavior) than to explicitly write the program. This is fundamentally altering the programming paradigm by which we iterate on our software, as the teams split in two: the 2.0 programmers (data labelers) edit and grow the datasets, while a few 1.0 programmers maintain and iterate on the surrounding training code infrastructure, analytics, visualizations and labeling interfaces. In most practical applications today, the neural net architectures and the training systems are increasingly standardized into a commodity, so most of the active “software development” takes the form of curating, growing, massaging and cleaning labeled datasets.
The process of training the neural network compiles the dataset into the binary - the final neural network. In Software 2.0 most often the source code comprises 1) the dataset that defines the desirable behavior and 2) the neural net architecture that gives the rough skeleton of the code, but with many details (the weights) to be filled in. cpp files) is compiled into a binary that does useful work. To make the analogy explicit, in Software 1.0, human-engineered source code (e.g.
0 kommentar(er)
