Posts

• Playing with DMX

During these times I decided to start playing with DMX. I bought a the Lumeri Wash 7.10. It has RGBW leds, 9 or 16 channels, and a moving head. It uses DMX512. The DMX in DMX512 stands for Digital Multiplex (protocol). Lights like this have a DMX input and output. so they can be chained. A collection of DMX devices is called a universe.

• Streaming to your TV

If you’re in quarantaine or in isolation, there’s a lot of staying inside. Perhaps you have to be in another room. Perhaps you just want to stream some online event to a larger screen. In either case, you want to figure out how to stream your desktop to your TV. If you happen to have a Chromecast, this is possible, but there are many ways to accomplish this. We will go through a few.

• Wasserstein and Gromov-Wasserstein

Suppose we have to come up with some kind of function that defines how different two probability distributions are. One such function is the Kullback-Leibler divergence. It is an asymmetric function: it gives a different value for probability distribution $A$ given probability distribution $B$ versus the other way around. It is henceforth not a true distance (which is symmetric), but a so-called divergence. A divergence also does not satisfy the “triangle inequality”: $$D(x + y) \leq D(x) + D(y)$$ is not necessarily true for all $x$ and $y$. It does satisfy however two other important conditions. A divergence is always zero or larger and the divergence is only zero if and only if $$x = y$$.

• Nonnegative Autoencoders

My intuition would say that a part-based decomposition should arise naturally within an autoencoder. To encorporate the next image in an image recognition task, it must be more beneficial to have gradient descent being able to navigate towards the optimal set of neural network weights for that image. If not, for each image gradient descent is all the time navigating some kind of common denominator, none of the images are actually properly represented. For each new image that is getting better classified, the other images are classified worse. With a proper decomposition learning the next representation will not interfere with previous representations. Grossberg calls this in Adaptive Resonance Theory (ART) catastrophic forgetting.

• Generating Point Clouds

If we do want robots to learn about the world, we can use computer vision. We can employ traditional methods. Build up a full-fledged model from corner detectors, edge detectors, feature descriptors, gradient descriptors, etc. We can also use modern deep learning techniques. One large neural network hopefully captures similarly or even better abstractions compared to the conventional computer vision pipeline.

• Attend, infer, repeat

A long, long time ago - namely, in terms of these fast moving times of advances in deep learning - two years (2016), there was once a paper studying how we can teach neural networks to count.

• Random gradients

Variational inference approximates the posterior distribution in probabilistic models. Given observed variables $$x$$ we would like to know the underlying phenomenon $$z$$, defined probabilistically as $$p(z | x)$$. Variational inference approximates $$p(z|x)$$ through a simpler distribution $$q(z,v)$$. The approximation is defined through a distance/divergence, often the Kullback-Leibler divergence:

• Machine learning done Bayesian

In the dark corners of the academic world there is a rampant fight between practitioners of deep learning and researchers of Bayesian methods. This polemic article testifies to this, although firmly establishing itself as anti-Bayesian.

• Inference in deep learning

There are many, many new generative methods developed in the recent years.

• denoising autoencoders
• generative stochastic networks
• variational autoencoders
• importance weighted autoencoders
• generative adversarial networks
• infusion training
• variational walkback
• stacked generative adversarial networks
• generative latent optimization
• deep learning through the use of non-equilibrium thermodynamics
• What is contrastive divergence?

In contrastive divergence the Kullback-Leibler divergence (KL-divergence) between the data distribution and the model distribution is minimized (here we assume $$x$$ to be discrete):

• Yoga 900 on Linux

The Yoga 900 is a beautiful machine that has a considerably long battery lifetime and can be folded such that it functions as a tablet. The Yoga arrived on Friday and the entire Crownstone team was enjoying how it came out of the box: it lifts up! If you’re creating your own hardware you suddenly appreciate how other people pay attention to packaging!

• A baththub in your autonomous car

Will you have a bathtub in your autonomous car? According to many the future is a socialist paradise. The autonomous car will change everything! We will be car sharing. We can change parking lots into a lot of parks!

• Are we welcoming to AI?

Imagine one of the first AIs coming online. What is it gonna read about itself? How would it feel? Would it feel welcome? What is definitely the case is that it will learn a lot about humans. This is for example what Musk is saying about this alien life form:

• Device recognition and indoor localization

We have put the Crownstone on Kickstarter, a smart power outlet with quite sophisticated technology. I think it’s nice for the general hacker community to get some more insight on the technology behind it.

• They will cry a thousand tears

Perhaps you have seen the recent TED video from Nick Bostrom. Here you see an extended talk from him at Google:

• Legendre Transform

The Legendre transform describes a function - in the normal Legendre case, a convex function (but for the generalized case, see [1]) - as a function of its supporting hyperplanes. In the case of a 2D function these are supporting lines. The supporting lines are the lines that just touch the function. These lines do not intersect the function anywhere else if the function is convex.

• What's the thalamus?

If you’re interested in how things work, our brain is one of the most intriguing devices around. I love reverse engineering stuff. Understanding limits and colimits within category theory can be just as rewarding as getting to terms with the intricate structure of the brain.

• Linux Graphics

It all started with annoying messages that nobody seems to understand (/var/log/syslog):

• Sampling of Dirichlet Process

Thousands of articles describe the use of the Dirichlet Process, but very few describe how to sample from it. Most often one is referred to Markov chain sampling methods for Dirichlet process mixture models (pdf) by Radford Neal (at University of Toronto), which is a nice piece of work, but still a bit dense as an introduction. I contacted him by email about certain things that were difficult to understand at first and he was kind enough to respond, thanks a lot! Definitely also check out his blog in which he regularly showcases his fast version of R.

• Bayesian approximation

In the world of Bayesian’s, a model is a triplet $p(x,z,\theta)$. The observations are random variables $x$. And then there is a seemingly artificial distinction between the random variables that are called hidden ($z$) and other random variables that are called parameters ($\theta$), and are hidden as well! So, how come that parameters got their distinguished name? In the case of for example a clustering task, we can assign each observation a corresponding hidden variable: an index of the cluster it belongs to. Hence, there are as many hidden variables as there are observations. Now, in contrary, we might define parameters in two different ways:

• Where are the aliens? Where are the dragons?

One night I was lying down staring at the stars and it dawned upon me that I was not alone. I had only a few of the many alien eyes. Just like them I was figuring out if my god existed. I felt part of this cosmic family more than anything before. Something bigger than our soccer team, our continental heritage, or our world wide scientific efforts. All these eyes… The universe becoming aware of itself.

• Google Glass - Translating Cat Meows

Interesting applications of Google glass? I encountered very few still. I think some creative minds have to sit together and go for it! Translations of foreign languages, and reading out loud for blind people, or the illiterate. Sure, two minutes of a creative session under the shower, and you will come up with such ideas. But what’s next? Do we really need to translate all people around us? There are so many annoying conversations! Perhaps the glass can assemble them to a nice creative story, or a poem! And of course, there is no reason to only use human input. A sound from an animal can directly translated in a warm male or female voice. The barks of your dog become “Hey! I see someone I don’t recognize!”, or “Dude, I am so hungry!”.

• Our future

Black Mirror, the first television series, really describing the future. The near future, a black future.

• Welcome

This website contains a few links of moderate importance to what I do. For my work see the company we started at Almende, namely Distributed Organisms (which we informally call DoBots). DoBots is a very exciting company which sells internet services for large groups of robots. In Replicator we did research with respect to self-reconfigurable robots, but its applicability is still far away. However, in FireSwarm we can actually use a group of aerial robots to find a dune fire as quick as possible. At times I might post some things about robot cognition, because the thing that I like (professionally) more than robots is artificial intelligence.

subscribe via RSS