February 20, 2019Jonathan Wagner9 min read
I originally had a simple goal: do a computer vision task solely from my browser. Having seen some very nice colour transfer effects before, I figured I could try a simple one. So I found a nice and popular algorithm: color Transfer by jrosebr1 and started analysing it.
I quickly realised that there might be a problem since it mostly relied on OpenCV which is a Python library.
The python-OpenCV library is quite impressive and allows plenty of advanced applications for minimal effort (such as image or video processing and object detection). The question was: can we do all that in the browser?
And then began my dreadful experience …
Don't expect OpenCV.js to do all the things you can do with OpenCV in C / C++ / python: it is not as advanced. So web computer vision is still a pretty complex task.
The tutorials on the official website help a lot but:
.js file as the ones used in the python algorithm.
The expected outcome is to be able to pick one colorful picture and a black and white picture (from your browser), click a button and after a few seconds see the black and white picture colorised: computer vision!
Here you have a choice,
The first step was to set up a basic webpage to be able to pick our input images and display the output image from our wonderful color transfer algorithm.
This wasn't the main focus of this experiment so I basically took the one from the RisingStack article and customised it:
This section shows the breakdown of the computer vision algorithm I chose to translate.
What's the LAB space you might ask? As Wikipedia puts it:
It expresses colour as three numerical values, L* for the lightness and a* and b* for the green–red and blue-yellow colour components.
I have —unsuccessfully— looked around for what this
astype('float32') does and if there is an equivalent in opencv.js but haven't found anything . Let's hope this doesn't break too much…
So far so good, but the tricky part is in the
Here as you can see I used
cv.meanStdDev. Why didn't I use
.std()? Simply because they don't exist in Opencv.js
So after some googling around, I found this post mentioning
cv.meanStdDev which seemed to do what I wanted.
I tested and printed the stats output for the source image and … I don't have the same values Not sure if it comes from the
But except for
bMean, the values are only off by a bit so I figured I could go on and see what the output images would look like.
Looks easy right?
⚠️⛔️ Well, that's because it does not work!
I looked a bit at what this LAB image was made of and this is the OpenCV matrix for the L part of the Lab plan of a picture of a sunset (the
b part are built the same with different values):
Without helper functions to do arithmetic operations, I don't think it's worth spending more time with this library.
My experience with computer vision in the browser using OpenCV.js has really not been great, but that doesn't mean you shouldn't use it!
There are some breath-taking demos of in browser computer vision done with OpenCV on their website such as:
So in my opinion, if what you want to do can be done in a couple openCV calls without additional manipulation, just go for it!
Look at the tutorials and hopefully, you'll find out how the functions you want to use work.
However, if you want to do more than what OpenCV offers out of the box, like manipulating pixels, I suggest you try and find another way of doing it (in a python backend for instance)
Web Developer at Theodo