Extracting the background and moving targets from a live video

Tuesday, November 27th, 2012

Recently I had been dealing with bugs and cells. I was trying to locate the moving cells Euglena with a camera. The Euglena is a single cell that belongs both to the plans and the animals. A picture of the Euglena is shown below. The length of a single Euglena is about 50 um.


The illumination was not good due to my poor optical setup. I tried to identify the cells according to its intensity and size but neither worked well. The strategy I took at last was extracting the stable background and then compare it with the live video, so that the moving targets can be identified. The way of generating the background is averaging all the grabbed images (or, video as we call them). The changing bits are then smoothed by the number of the frames.

When averaging the images, we assume the mean of N framesimages is A_n and the (N+1)th frame is I_(n+1). Both variables are 2D arrays. Then the mean of (N+1) frames is
A_n+1 = (N * A_n + I_(n+1))/(n+1)

The code is shown below (with re-calculate/ clear function):

This SubVI can be called without external shift registers.  This simple function allows us to extract the still background from the video. And thus the moving (or any changing) targets can be extracted no matter how messy the background is. The result is shown in the video below:

As I said in the description of the video, “This demo shows using an algorithm tracing an Euglena in the dish with poor (non-uniform) illumination. The mid-left and mid-right videos are raw videos from the camera. The bottom-left video is the background generated from the video in real-time. The bottom-right video is the target (Euglena) extracted from the video. The top-left video is the coords of the Euglena.” We can see that the Euglena were identified from the video even the illumination is non-uniform and the background is a bit messy.

Days of PhD (6), (7) and (8)

Thursday, October 11th, 2012


Here are the comics of ‘Days of PhD’ 6, 7 and 8. The character was changed, abstract and chatty.

Hope you will like it. The author signature @科学玩家 was my ID on weibo.com and now changed to @vanja .

Creat Region of Interest mask in IMAQ image

Wednesday, May 2nd, 2012

I have been working on using a USB webcam as a research tool to observe the heart beat of a Daphina (water flea) these days. Since LabVIEW 2009 IMAQdx is available for 3rd party USB cameras, no extra USB driver is required. So I used the old but classic webcam Philips SPC 900NC to build a Daphnia observation system. I may talk more about this later.

To cut a long story short, I tried to process only the Region of Interest (ROI) of the grabbed image rather than the whole one. This can make the analysis easier. Since this USB webcam does not support ROI imaging, we cannot set XY pixels on it.   I tried to google the solution and found creating a mask or, a pseudo ROI, can block the unwanted image. I programmed an example for blocking the unwanted image by selecting ROI on the image:

For some unknown reason I cannot create snippet from this VI, so I save the screenshot instead.
In this example three memories were created to store the original image, the mask and the modified image. If an ROI is selected (FALSE case), firstly convert the ROI info to a mask, and then mask the source image with the mask image. If no ROI is selected (TRUE case), the destination image is linked to the source image.

Note that:1. The property node “ROI” is created by right clicking the “image” indicator in the block diagram and selecting “Create>Property Node>ROI”.
2. The image type of the “Mask” must be Grayscale U8. The original and the changed images can be any image type.
3. It is masking the image rather than ROI imaging. So it will NOT increase the frame rate nor decrease the image size. All black regions are filled with 255 by default.

Hope this example can be helpful. Let me know if you have any problem or are interested in controlling conventional USB webcams with LabVIEW IMAQdx.

Preparing for LabVIEW CLD exam (2)

Saturday, September 10th, 2011

I should have written this post one month ago before I took the CLD exam. The good news is I passed that. 🙂 80 out of 100, which is more than I expected (I’ll carry on this later).

I will not talk anything about the exam itself, as I promised in the exam. But just finish the things I prepared for the exam.
1. I found this blog “Pass your CLD/CLA exams the JKI way” very useful, which gave me the confidence of carry on preparing for the exam. Though I didnt use it at the end, it is very helpful for preparing your own template.
2. Timing problem. The timing problem is the tricky part in the exam, such as how to pause/resume/restart your state machine. I made a subVI beforehand to practise it. You can find it here.
3. Pay attention to the documentation, which should be the easiest part to get the credits.
4. If you can finish the sample exams within 4 hours, I’m pretty sure you can pass the exam. If you can finish that within 3 hours, you can get the full marks in the exam.

Since I didn’t finish all the functions during the exam, I felt bad about it. But it turned out I got almost full marks in the programming style and the documentation parts, which saved my life. I didn’t have the plan to take the CLA before, but now I’m considering about it.

Good luck to all of you who are going to take the CLD exams.

Related post: Preparing for LabVIEW CLD exam (1)

Pause and continue the time with Elapsed time.vi in LabVIEW

Sunday, July 31st, 2011

Elapsed Time.vi is a very useful Express vi which returns the elapsed time and can be reseted. Unfortunately there is no pause function in Elapsed Time.vi ( LabVIEW 8.5 to 2009). I had been looking for the examples of pausing and continuing the time in LabVIEW, but there seems not be a simple one.
So I created this myself. It is not efficient in term of excecution time, but it is simple to be made from scratch and memorized.

Basically the shift registers are used to store the previous elapsed time. When it is in the false (run) case, the elapsed time is added to the ‘total elapsed time’. When the vi in the true (pause) case, the total elapsed time is transferred to the start time and the Elapsed Time.vi is reseted (stopped). Hope this will be helpful. 🙂


———————–In response to Mark’s question——————————–

So I added the reset function for this SubVI:

When you reset the clock, you simply ‘reset’ it and send ‘0’ to elapsed time. Hope it helps.

Scrolling the mouse wheel to change values in LabVIEW

Sunday, July 3rd, 2011

This is a simple demo using the mouse wheel to control the numeric indicator in LabVIEW. The value of the ‘mouse wheel position’ increases or decreases when we scroll the mouse wheel forwards or backwards.
The reason of posting this is when I tried to using the mouse wheel to control a camera (to zoom in/out) in LabVIEW, it took me a lot of time finding the right example. If this post happends to be the top 3 of your google results, it should save you some time.
This VI was modified from the example ‘Basic Input Demo.vi’ in LabVIEW. Since the windows’s dll only returns the moved (or related) position of the mouse wheel, a shift register was employed to display the ‘absolute’ position.

Sorry that I cannot upload the .vi file for the limitation of wordpress.com. But it is easy enough to re-create this vi yourself from the snippet.
Note that:1. There is no ‘mouse wheel’ triggering event in the event structure (at least before LabVIEW 2009);
2. The .dll file in ‘Acquire input data.vi’ returns only the moved position of the mouse wheel comparing to the position when the dll was called last time;
3. The returned mouse wheel postion was divided by 120 in this demo, which may change according to different mouse setups.
4. The polling event (nodes in the while loop) could be put into the timeout case if you want to use event structure together with it (and the timeout shouldn’t be -1).

Preparing for LabVIEW CLD exam (1)

Tuesday, May 31st, 2011

So I registered for the CLD (Certified LabVIEW Developer) exam on 5th August. I knew LabVIEW when I started my Ph.D in November 2007 and have been using it for 3 years. I had been “learning” LabVIEW for 2 years. In the 3rd year I was more of using it instead of learning more things. Since it is said that the CLD is suitable for users with experience of “a year and a half”, I think it is fair for me to take the exam.
There are 4 exam samples online (link) and they are said to be very similar to the actual exam. Also Darren’s note on CLD is very helpful (link). So my plan is taking the examples as the real exam and finding the knowledge I’m not aware of. One exam lasts 4 hours and I have to use one weekend to finish one. So the examples have to be finished before end of June (5 weeks to go).
I took the “ATM machine” examle last week and failed to finish it in time. I think that might be the most difficult one among the 4 (or not?). Anyway, fortunetly there is still time to fix that problem.

Here are what I have learned from the examples, and I will add more thoughts as the practice goes on. Hope this can be helpful:
1. We do not need to install any module beside LabVIEW to take the exam. It tests your coding ability instead of your module experience.
2. The logic of the exam is very complicated.
3. For me, the exam could be finished if more time (maybe 2 more hours) was given. Thus the challenge is how to finish it in the limited time. You cannot do that without good practice.
4. Use state machine, as stated in the exam. And I found the queue operation is not necessary in these 4 exam samples.
5. The project manager is not neccessary for there won’t be many versions of your code.
6. Folders named “SubVI” and “Control” are useful. Put the Top_level.vi in the top folder.
7. There could be file read/write operation, and thus we need to master how to manipulate the file path and text-related functions.
8. Draw the states clearly before coding, which saves you a lot of time later (Or, do not code before the logic of states is clear).
9. Because of 8, pens of different colours are very helpful (but is it allowed to bring our own pens?).
10. Set the preference of block diagram a ‘comfortable’ way. E.g. untick Display terminals as icons; untick Show dots at joints; untick Auto wiring; tick Default SubVI terminal as required.
11. Quick drop saves you time.
12. Save the code frequently in case of unknown situations.
13. If you cannot finish the code, create all SubVIs only with front panels. At least the program could be ran (with some missing functions).
14. Knowledge of timed loop and control reference are necessary (Update: I added an example in this post).

FYI: this is my drawing of the state machine. It will be messier if there was just one colour.

I will carry on the exam simulation this weekend and my timeline is: finish the state diagram within 1 hour; Create the structure and SubVIs with only terminals and descriptions in 1 hour; try to finish all the functions in 2 hours.

Good luck to me.

Update: Related post Preparing for LabVIEW CLD exam (2)

Days of PhD (5)

Monday, April 11th, 2011

Sorry for the late post. So I have been writing thesis these days (fingers crossed) and did draw some more comics just did upload them. Now here is one of them. Hope you’ll like it.

P.S. I found that there is a lot of potential work for my academic writing. Talk is totally another issue. Good luck to me.

Making the number of shift registers programmable

Thursday, January 27th, 2011

Shift register is very handy in LabVIEW programming. To add one more shift register (say, shift one more time), what you need is only click and drag down the shift register. If you want to recall the data generated in a loop 5 times ago, you drag down 4 more shift registers and pick the bottom one.

But the drawbacks are obvious,
1. It is not neat nor scalable in coding.
2. User cannot modify it without modifying the code.

So here is a alternative method for large/unstable number of shift registers — using the queue in the loop. The thought was quite straightforward, just to find something that can buffer the data. I’m sure there are many cleverer ways.

So the queue is used to store the data. With enqueuing and dequeuing we can do the shifting. The case structure is to determine if we shifted the right number, and the condition can be modified (e.g. if(get queue state==#shift regisiters) ).

The problem is solved now. Please let me know if you got better solutions:)

7 sins of IMAQ module

Monday, January 10th, 2011

I have been using IMAQ module in LabVIEW these days, and the integration of many algorithms saved me a lot of time. It is the core module when we want to do Machine Vision. I haven’t digged into it since the subset seems OK for my job. But in this post I’m not saying the beauty of it, instead, I’m going to tell seven sins of the IMAQ module. Just to clarify, I splitted some up in order to make it “7”. All the problems are based on vision development module 2009.

1. It has to be in IMAQ. Sometime I just want a simple “2D array filter” to proceed my 2D data, but there is no such a node in the core functions, although I though it’s not complex. And the fact is, if I want to proceed my raw 2D array in any ‘image’ way, I have to buy and call IMAQ module to do that, plus conversion back and forth. That’s not handy.

2.  Non-standard connector layout. Here is the connector  layout:

It’s not using standard 4224 layout pattern, and the wiring becomes a chaos in this case.

3. Old style front panel. Here is the front panel comparison of the “IMAQ create.vi” and “Open/Create/Replace file.vi”

As you can see, the controls and indicators are different in two VIs. So if we have IMAQ VIs with other VIs in the same top level, we will have diffrent front panel controls if we just create controls by right clicking.

4.IMAQ VIs should support all image types which makes sense” as described in the NI idea exchange.

5. Along with the last sin, there is no boolean array input/ output in “IMAQ ArrayToImage.vi” and “IMAQ ImageToArray.vi”. Actually boolean 2D array is quite useful in many pattern recognization applications. (Yeah, this is the extra sin I just divided from last one)

6. “Image Dsc Out” missing in some VIs. E.g, after the proceeding of the image and we want to convert it back to an array, do we need to release the buffer we created? If yes, there is no such a connector on the output. I assume all the VI without the “Image out” connector is to be used as indicators, but still, it’s better to keep it uniform.

7. We have to create buffer to manipulate images. In this module it’s not as smart as in Array functions, we need to create one buffer to proceed an image, mutiple buffers for mutiple ones (or, use one buffer one by one). It would be nicer if NI can make this easier.

Anyway, IMAQ is a very useful module and hope I’ll study ultilize it better in the future.