Marking each product with a unique ID is becoming a standard across industries. The opportunities for tracing, statistics and even predictive maintenance are practically endless. Before doing this, a manufacturer needs to choose between two major options: standard human readable text or some sort of visually coded information like QR or Data matrix code. I will try to argue in this article for the superiority of 2D barcodes over standard text and use Data matrix as an example.

After the code is marked, it has to be read by people, machines or both. Choosing Data matrix code (DMC) eliminates the option of human reading but still could save a lot of time and ensure long-term product traceability. Here are a few reasons why.

Error correction/Robustness

An item is usually marked at the beginning of the production cycle and needs to go through series of different phases which are often aggressive and might involve corrosive fluids which might damage the marking. If we have a number 8 written in some industrial font, in case of a damage of some parts of it, it could easily become 3 or 0. If the damage is positioned perfectly, it is impossible to tell apart the damaged number 8 and the intact number 6. This situation could be avoided by choosing a special font, but it still leaves the problem of the error correction unsolved. Considering this, it is obvious that error correction is an important topic in unique ID recognition.

Data Matrix has the ability to detect and correct damaged information. In the last column of the table, you can see maximum correction values for different DMC sizes.

Space/Shape

Since Data matrix code has an alignment pattern, it is possible to precisely position where the data is written. Combining that with the error correction results with much less space used compared to text and still yielding better reading results.

Marking speed

If we are talking about inkjet or laser printers, there is very little difference in marking speeds between text and Data matrix. If on the other hand the marking needs to be engraved or punched on the surface, speed becomes an issue. Let us take a look at example of ten numbers written in Dot Matrix font which is often used for food expiration date. Using this font, we would need 124 holes for to mark “0123456789”, and only 86 for the Data matrix code. This discrepancy will increase further with the greater number of characters.

Encoding

If the marking doesn’t need to be read by people, it could be coded, meaning there is a possibility to pack more information within the same number of characters.

Let us say you want to store dates for the next 20 years, between 01/01/2021 and 01/01/2041. If standard alphabet is used at least 6 characters would be needed. If the data is encoded, only 4 characters are needed. For example:

0000 => 01/01/2021

0001 => 02/01/2021

0030 => 01/02/2021

…

7305 => 01/01/2041

Conclusion

The only obvious advantage of having a standard text written on the product is readability by people. But if the numbers are going to be read by people, that implies manual tracking and data entry which automatically means it is prone to mistakes. This problem can be addressed by buying affordable hand-held readers or even mobile applications. Of course, if the conditions allow, it could be possible to add the DMC additionally to existing human readable text. In my experience whichever reading rate is achieved by using optical character recognition, it will always be surpassed by the Data Matrix code.

Visual classification is a simple and self-explanatory  process, at least for humans. Machines, on the other hand, have a much tougher time classifying objects in the images. Hence, classification has become one of the most engaging challenges for computer vision engineers. Whether your task is to distinguish between good and faulty products or to categorize facial expressions, similar methods can be used.

This article will showcase a basic, yet challenging and relevant classification problem (also delicious). The goal is to classify different sorts of fruit and vegetables. There are 10 distinct classes as shown in the image below.

The classic approach

The first step is to get familiar with the dataset in order to better understand the challenges of the task. It is also a good idea to write down the differencies you have noticed while looking at different classes. The two main features, not just in this case, are the color and the shape of the object. While the shape varies based on the perspective, the color can be influenced by lighting conditions. To learn more about robust color detection, check out the following article.

Another useful feature is texture. Imagine a task where you have to differentiate between a peach and a nectarine. Since they are the same color and they have the same shape, the distinguishable difference is that a peach has a rough surface while nectarine has a smooth surface.

Features mentioned above are used in the classic approach to classification. Nowadays, this is a common practice, so programs like Halcon have a built-in identifier functions to speed up the process. Halcon's identifier is based on the color and texture features. It managed to correctly classify 69/70 images, with 1.4% error rate.

What happens if the conditions are not perfect? One of the potential uses of this concept could be at the fruit/vegetable weighing stations in the grocery stores. In that case, the fruit and vegetables are in a plastic bag, which can significantly interfere with texture detection. The plastic cover was added to simulate the plastic bag and, in this case, the identifier manages to correctly classify only 60/70 images, with 14,8% error rate. This is not good enough since most classification tasks require error rate lesser than 5%.

Deep learning

Another popular approach to classification is deep learning. It is a machine learning technique based on artificial neural networks which are designed to mimic human brain. They can learn from data without specifying which features to look at. For example, when humans see a lemon, they do not really think about its color and shape, they simply know it is a lemon because they have seen it many times before.

Therefore, deep learning is a really powerful tool which also requires a certain computational power while training the model. On the contrary, it is not that difficult to implement, since there is a lot of deep learning libraries which do most of the work for you. Similarly to the previous method, it only requires a correctly labeled dataset, with optional preprocessing.

Deep learning method was applied on the second datased, with a plastic cover on the fruit and vegetables and it correctly classified 70/70 images, with 0% error rate.

In conclusion, comparing apples and oranges is not as easy as it sounds, unless you are using deep learning.

If you haven’t already I recommend you read the first part before continuing with this article further. In the last article, an idea for non-linear project management was introduced. Looking at the image below, we can see that the R&D project workflow looks a lot like a version control tree mostly used for software development projects. Since Git is the most popular solution for version control, I will refer to it in the text as just – Git. In this article, we will take a look at which Git concepts could be used as research project management principles.

We should not give up tickets

Every curve looks like a straight line if you zoom close enough

Before discussing which parts of standard project management are not optimal for research, let’s talk about the ones that work. Even though the workflow of a research and development project is not linear, one unit of time (a week or a sprint) is. Kanban and to-do lists are very intuitive for a short-term focus. The first thing most of us do when presented with a task is to write a list on a blank sheet paper or a digital document. The problem lies with the bigger picture where those tickets do not represent a real situation of the project flow. 

Ticketing systems are excellent for short term progress tracking but lose value on marathon projects. Ideally, just by using a standard ticketing system with a couple of additional tags, a Git tree should be generated automatically.

Branches

Typical Git workflow has one main branch called the master branch, but this might be the only concept that cannot be directly copied. There is no possibility for a single main branch if there is more than one topic in the project. Project branches should be used for different topics and they will probably never merge. There is one exception discussed in the next paragraph.

Not every dead-end is a loss

Deciding when to close a branch requires a topic on its own. It might be possible to say that an idea is never a dead end and an implementation will come eventually. But a dead-end in one branch could be a solution in another and this is a great breakthrough that should be documented. We propose something like this:

This is a rare case where Git-merge might happen. An alternative solution could be cherry-pick, if we wish to keep the branches separated at any cost. 

Other stuff

I’m sure we all suffered from a tunnel vision where we focused too much on a problem and ignored everything else that could have helped. Imagine a situation where you’re working on a problem that after changing some minor details requires 30 minutes of testing, and you’re aware that the procedure could be significantly optimized. I personally had that kind of situation and at that moment optimization didn’t seem like a rational solution. It would be very interesting to track the amount of the time invested in “other” tasks and their influence on the project performance. My gut tells me that those investments lead to a significant speed-up of the findings.

If the mentioned test could be optimized to one minute, that would mean 30 times more tests. Just last week I received an IM from my employee telling me he sped up the test algorithm 85 times. Initial versions are often hasty written (as they should be) in order to test a concept. But a balance between optimization of an idea that has will fail nine out of ten times and using the first draft for the next three months needs to be found.

In the PM tools, there should be a reward system that would let you know when you’ve “earned” the other stuff time e.g. for every nine days of work on the core problem, you receive one day for optimization. The relationship between the time invested and the benchmark curve would be something interesting to see.

Benchmark timeline

As already mentioned in the previous article, having a benchmark keeps us on track and motivated. Connecting the Git-tree with benchmarks could also give us the idea which branches worked, and which didn’t. At this stage, it is hard to tell exactly which benefits this would offer to the project manager, but I’m sure this will be clearer later.

Conclusion

Looking at the big picture, the R&D project has plenty of similarities with Git-structure and there are plenty of concepts that could be replicated. I would like to hear your experiences from managing or being part of research projects. Feel free to contact me with your opinion.

Let’s imagine a situation where you want to speed up the production line which has plenty of cameras for visual inspection which work “on the fly” – without stopping the product for inspection. Could the existing camera systems continue working without any modifications? Possibly, but let’s go through the specifics.

To achieve system feasibility, we need to ensure that the camera and lens parameters are balanced. Two main things that will be affected are exposure time and depth of field. If the production line speed increases and exposure time stays the same, there is a possibility of seeing a motion blur on our images. Exposure time, in this case, is the time component in the path covered by the object in the image.

The next logical step would be to decrease the exposure time to avoid this problem. We could either decrease the time on the camera or increase the physical opening of the lens – aperture. Opening the aperture will in fact cause more light to fall on the chip, but it will reduce the depth of field. Depth of field is the distance between the nearest and the farthest objects that are in acceptably sharp focus in an image. 

Why does the opening of the lens decrease the depth of field? An explanation is quite simple, having a smaller pinhole means that only direct rays of light fall on the sensor. As we open it further, reflected rays are coming more and more, which translates as a blurred image. In the cover image, we can see that only a few rows of the cookies are sharp, which means they are inside the depth of field. If you wish to explore this further, I recommend this article.

Let’s explore the exact relationship between the (production line) speed, aperture, and depth of field with a 12 mm lens.

To keep the same exposure time, which effectively means the same distance traveled during the exposure time, we have to move somewhere along the gray curve.

If none of these options satisfy our requirements, we need to consider a lens with another focal length. Comparison between 8 mm and 12 mm lenses can be seen below.

Changing the lens has its consequences, especially on the field of view – choosing a smaller focal length will give us a bigger field of view, which automatically means a smaller number of pixels per millimeter:

One more option would be to increase the light intensity if that is even a possibility. Of course, there is a chance that the initial depth of field was large enough that the speed increase will not have a significant impact on the vision system performance.

There is more than one solution to this problem and the purpose of this article was to give you a general overview of balancing between the different camera and lens parameters. To make an exact calculation use online calculators like Vision Doctor or contact us directly.

It is possible to take many different project management courses online or offline, but all of them, when it comes to the topic of research and development will only glance at or ignore it completely. This is how the PM course would like you to imagine a project:

This is how a small part of an R&D project actually looks like:

How to even start managing something like this? Here are some guidelines that helped me personally over the years, as well as some general advice.

Clear cut-off points

A few years ago, I was working on a high precision measurement project that seemed like it had dozens of potential solutions, but none of them led to any progress. My boss at a time told me one simple sentence that had a significant impact: "Ignore all problems that have an error smaller than 10 microns for the time being".

That advice helped me focus on the 20% of tasks that were relevant and ignore the minor details at that moment. Everyone knows that 20% of work delivers 80% of results, but to define them during the R&D phase is a tiresome chore.

Don't get emotionally involved

This is probably the most challenging point to adhere to. We all like to believe that we are rational individuals who make their decisions based on facts and meaningful arguments. Still, there are whole fields of science arising in recent decades that prove that we are not i.e., behavioral economics. It is tough to give up a beautiful idea that you've spent weeks or maybe even months perfecting and written 10.000 lines of code, but if it is proven to be wrong, you have to give it up, and you know it.

The question is how to deal with the death of something you've created. In my experience, you should write. If the idea is proven wrong, you should at least write down what you've learned so you can apply it in the next situation or to remind yourself not to go that path again. Worst case scenario you'll be able to justify your time with an excellent report to your superiors or clients.

Write an R&D journal 

Every day you have to test a large number of different hypotheses, which may only slightly differ from each other. This will lead to confusion unless the critical points of the day are written down.

I will assume that if you're reading this piece, that you've read plenty of professional articles where you've been told to write a journal, and probably thought of it as childish. I know I have, and I've entirely missed the point. 

If you're working in R&D, there is a huge chance, you're a STEM major, who rarely needed to write something coherent while studying and dismissed writing as something reserved only for liberal art majors. Here is how it's useful to science. Writing is slow and hard, it requires to focus on one subject, and to me, it is the complete opposite of my thinking; quick and scattered. Taking ten minutes to write something at the end of the business day is a form of meditation.

It is an excellent method to sort out your thoughts and to remind yourself of bad ideas and why they didn't work out. I like to take ten minutes before leaving work to write the key points that have been tested today and why they did or didn't work. This gives me time to pause and ask myself what I am going to work on tomorrow.

Since I've started a project journal, I've consulted it on many occasions to see what I did in the past. I find it much more practical than scouring through my emails and reports.

Build benchmark tests and simulations

Create a relatively large test sample with fixed parameters on which new ideas could be regularly tested to follow your progress over time. If possible, automatize everything, and save in a way that will enable comparison later.

This step might seem obvious, but at this very moment, I realized that it took me six months to build a quality benchmark test on one of my recent projects.

Automate your reports

A few things in life demotivates as quickly as doing the same thing for the hundredth time. I'm positive that you're familiar with that feeling. You've done the hard work, and now you need to display your results and create a report. You're tired, and you don't want to write another procedure to do it automatically. Instead, you open the report and edit the missing commas or extra columns manually.

It is hard to tell when it's time to automate something, and when are you just wasting your time writing a routine that may only be used once. That's why I use a rule of thumb if I did it more than twice manually; I'm going to spend some time in automatizing it because there is a high chance that it will come up again. The reason for that could be found in Lindy effect best explained in Nicholas Taleb's book Antifragile:

If a book has been in print for forty years, I can expect it to be in print for another forty years. But, and that is the main difference, if it survives another decade, then it will be expected to be in print another fifty years. This, simply, as a rule, tells you why things that have been around for a long time are not "aging" like persons, but "aging" in reverse. Every year that passes without extinction doubles the additional life expectancy. This is an indicator of some robustness. The robustness of an item is proportional to its life!

Using the same logic, we could say that if the procedure was done manually two times, it will be done again two times. Don't immediately invest three days into it, start small and build up from that. Most of the time, only 20 minutes will do the trick.

Set a day for the "other stuff"

You should set yourself a day or half of the day each week for all the things you're too busy to do during the rest of the week. At least for me, this is Friday afternoon. Office gets emptied sooner than other days, clients are calling less often, and I'm able to concentrate. It would be best to keep a list of small improvements (e.g., optimizations, report generation) that need attention but aren't priorities on a current project or your client's preferences.

Making small regular improvements is much more powerful than waiting for a carefree day and doing it all at once. The truth is that you're probably never going to have a massive chunk of time to finish your side project or the "other stuff". Even if such time were to come to our companies, we would surely find a way to waste it. 

Do it right the first time

It took me a while to realize that the time when I'm going to have a set of tasks, do them in the required time and go home is never going to come. This perfect balance when there are exactly 40 hours of work per week is probably not even theoretically possible. Almost every day, there is an exception to this rule. A client called; meeting took longer than expected etc. The only viable solution is to manage the chaos because it is not going to go away. If it does, the company is probably not growing anymore. 

There are rare cases where a workaround should be used because of a lack of time. Yes, you will frequently be pressured to do it quick and dirty. If you don't want to encounter the same problem in three months, where you now have to change the hardcoded solution to fit another client or adapt the existing solution to be dynamic enough for multiple clients. If a scalable solution was done the first time, the second problem would barely exist. 

I have yet to see a situation where a piece of code which I wrote well for the first time (meaning it could be applied to more than one client, easy to read and debug) that it didn't pay off in the long run. Sometimes I had to wait a whole year for it, but it always did!

A great answer on StackExchange inspired this post, and if you have something to add, feel free to write a comment or contact me directly via LinkedIn or mail.

Here you can find a second part discussing using Git's solution to mentioned problems.

Background estimation is a common problem in image processing that, in some cases, could be resolved by using a simple method. If you imagine a highway with cars passing by, most of the time, the background is visible (e.g., road, trees, traffic signs) with an occasional "disturbance" in the form of a vehicle. These outliers are usually easily solved by using a median filter.

Implementation

To find a background image, we need to calculate the median of each pixel in a specific window e.g., 50 last images. This could be done by stacking images in a 2D array and calculate the median for each pixel. However, most computer vision libraries already have a ready-made method like rank_image in Halcon. 

The results are shown in the first video. Upper left corner shows an original video, upper right is the basic median method, bottom left improved method discussed in the next paragraph, and the bottom-right absolute difference between the original and the improved method.

For the most part, results are impressive, but if you look closely in the latter part of the video, some artifacts are created on the top of the road, because the background is rarely free from cars. This method doesn't work for heavily congested roads, at least not for the whole image. A slight improvement can be achieved.

Improvement

Instead of adding every pixel to our buffer (image stack), let's add only the ones where the change between neighboring frames was less than X (some arbitrary value e.g., 50) in pixel intensity. Instead of adding their values, we could put a random number between 0 and 255. This results in an even distribution until pixels with similar values arrive. In the original algorithm, the intensities could group around the colors of the vehicles before the road went free. The salt and pepper-like noise can be solved with a simple median filter, but it will not be used in the video, depending on the application needs. The difference between the original and improved algorithm is not very visible in the first video (upper right and bottom left corners), but if you pay attention to the middle lane in the next video, differences are quite visible. As promised, the algorithm does not work for the congested lanes.

To emphasize this difference, in the third video, you can see histograms of a single pixel in the middle lane, and a jumping red line, which represents a median of the stacked images in our chosen window. The second median becomes stable much faster at a value that is approximately the same intensity as the expected background (road).

Further improvement

There is one more interesting phenomenon happening in the third video; instead of one normal distribution in the histogram, we see two different peaks. Our median does not end up in the middle of a larger one. The solution for this problem might be taking the most common element in the histogram instead of a median, but this will be a topic for another article.

The main reason why there are so many topics related/dedicated to lights and cameras on our page is that I believe programming should always be the last resort for solving a problem. Choosing a correct light or camera could save countless hours of coding or even save a project. Coding is fun, but wasting time on low-quality images is often just that – wasting time. This article is written from a computer vision perspective, but it can be applied to a bigger picture. 

There is rarely only one way to solve a problem, and often rewriting functions or retraining neural networks is not the solution. The goal of this post is not to diminish the importance of programming on the project, but it is much easier to build a reliable system when all external prerequisites are met. Six essential questions should be answered: 

1.     Is the correct camera sensor being used?  

•  Should RGB or monochrome camera be used? 
• Is the camera resolution adequate? 
• How does the pixel size influence the image quality? 

2.     Should another lens be considered? 

• Are the inspected objects sharp on each working height – “Depth of field”? 
• Does the image look warped on the edges – “pincushion or barrel distortion”? 

3.     Is the lighting appropriate for this application? 

• Could particular objects be filtered or emphasized by using another light color? 
• Does the light cause overexposure, and could that be used to advantage? 

4.     Is there interference from the environment, and could it be reduced? 

• If possible, natural light should be eliminated. 
• Are shadows occurring during some part of the day? 

5.     Are any filters available? 

• Could your problem be solved by using a polarization filter? 
• Are inspected objects better visible in another part of the spectrum (e.g., infrared)? 

6.     Cabling and power supply rarely cause problems, but when they do, it can be painful to detect them. 

• Could the bending of the cables cause signal interruption? 
• Inadequate power supply. 

Only if each of these topics has a satisfactory answer, should we turn to programming. Each of these topics is going to be discussed individually, but this post should give you a general idea of how to build a reliable vision system. 

For a more general perspective on this idea check out Jeff Atwood’s blog entry The Best Code is No Code At All 

If we can help you with any of the mentioned topics, feel free to contact us at info@subpixel.hr. 1-hour consultation is free of charge. 

Overexposure is the effect when optical sensor reaches saturation and from that point on, only the maximum pixel value (usually 255) is shown. Overexposed objects can cause a lot of headaches in image processing, but there are ways in which it can be made useful. 

For OCR, bar code and similar applications it can be useful to overexpose the image to eliminate the noise or dirt, since the printed part is usually darker than the impurities. In the image bellow we see a printed text scribbled over with a graphite pencil.

If we were to try preprocessing the image with a filter or histogram equalization, results would not be satisfying:

Instead we could gradually increase the exposure until the scribbles are eliminated:

There might be a need for adjusting different exposure for every line, but the results are quite convincing. One fact should be taken into account when using overexposure and that is that object will change their edges and become "thinner" like we see it in the next image.

From this image it is quite clear that overexposure should be avoided for applications that require precise measurement.

Extension methods are a great way of advancing an existing library without modifying the original class. But it is possible to do something even more powerful with it. Even the dlls of a closed source commercial library can be extended. Halcon is a commercial computer vision library which will be used for showcasing this application.

HImage is a class available in the mentioned library which allows reading, creation and manipulation of images as well as extraction of some features. Even though Halcon has many features used in computer vision, sometimes it requires some customization. For the purposes of adapting exposure time, it would be practical to have the number of overexposed pixels. The end result would be a method name shown in IntelliSense with a dashed arrow like in an image below.

The implementation requires modifier "this" to be added before the first parameter and the method has to be static in a static class.

For a deep dive in extension methods you can see the free preview of Jon Skeet's book C# in Depth

Reliably detecting colors can be a challenging task when the shape of the observed object is curved and/or has inconsistent light from all directions. In the photo shown we have dices for which we can say safely that the colors on each side should be very similar. If we were to inspect the RGB colors of left and bottom sides of the first dice (1 and 3) the results would be very different.

Values where the black dots were excluded from the evaluation were:

Side with 1 point: RGB (248, 215, 201) Side with 3 points: RGB (123, 93, 81)

Using RGB color space is great for building cameras and television sets but not a perfect tool for telling the colors apart. A better way would be to convert the image to HSV (hue, saturation, value) color system. HSV separates the light intensity (value) from the other two dimensions (hue and saturation). Hue scale goes from 0 to 360 (degrees), and saturation and value from 0 to 100.

Color values using HSV space:

Side with 1 point: HSV (17, 18, 97) Side with 3 points: HSV (17, 34, 48)

Let us now show the intensities separated by channels:

First image shows intensities of each RGB channel and the second one each HSV channel. Looking at the first image in the second row (hue), we can see that the transition between the sides is very hard to detect meaning they are in fact the the same hue/color (17). There are some differences in saturation channel (18 vs. 34) and as expected the largest difference in the value/light channel (97 vs. 48).

If we were working on a task where similar colors should be clustered together we could use a color space like CIELAB which would give us the following result for the same surfaces:

Side with 1 point: LAB (241, 125, 143) Side with 3 points: LAB (174, 129, 146), where the L represents the light dimension.

Of course, this method has its limitations. Overexposed parts of the image cannot be reliably converted to "real" color because the information is irreversibly lost because of the sensor saturation.

Because you know exactly what you're going to get if you use vectors.

Starting with version 12 Halcon introduced vectors that can either contain tuples or other vectors. This allows us to use them as structures and in this case categorize donuts:

Data.at(0).at(0) := ['UID', 1]
Data.at(0).at(1) := ['Frosting', 'Chocolate']
Data.at(0).at(2) := ['Topping', 'Sprinkles']
Data.at(0).at(3) := ['Hole', 1]

Of course, this input could be replaced with a simple function:

add_new_object (Data, 2,  ['Frosting','Topping','Hole'], ['Strawberry','Sprinkles', 1], Data)

Properties := ['Frosting','Topping','Hole']

add_new_object (Data, 3,  Properties, ['Chocolate', 'None',      1], Data)
add_new_object (Data, 4,  Properties, ['None',	      'None',	     1], Data)
add_new_object (Data, 5,  Properties, ['Raspberry', 'None',	     1], Data)

...

Structuring data like this allows us to have a clear overview of data...

...and writing of simple queries:

** get_all_property_values (Data, PropertyName, OutputValues)

convert_vector_to_tuple (Data, Result)
tuple_find (Result, PropertyName, Indices)
OutputValues := Result[Indices+1]
return ()

** get_all_property_values (Data, 'UID', IDs)
** Result:
** [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

Like getting all of the possible donut toppings:

** get_all_unique_property_values (Data, PropertyName, OutputValues)


get_all_property_values (Data, PropertyName, ValuesAll)
tuple_sort (ValuesAll, Sorted)
tuple_uniq (Sorted, OutputValues)
return ()

** get_all_unique_property_values (Data, 'Topping', Toppings)
** Result:
** ['Chocolate', 'None', 'Sprinkles', 'Sugar']

Or getting donut indices (starting from one from top left corner) with chocolate frosting:

** get_all_where_property_equals_X (Data, PropertyName, X, OutputValues)
get_all_property_values (Data, 'UID', IDs)
get_all_property_values (Data, PropertyName, Frostings)
tuple_find (Frostings, X, Indices)


OutputValues := IDs[Indices]
return ()

** get_all_where_property_equals_X (Data, 'Frosting', 'Chocolate', OutputValues)
** Result:
** [1, 3, 8, 11]

You can find the sample code here: Example

Before trying to solve the problem using software, we should first reconsider if we're using the right hardware. This article will focus exclusively on the camera but for a more general article take a look here.

Considering that prices for mono and color cameras are identical for some manufacturers, does it still make sense to even consider buying a mono camera for your application? If you are just looking for a quick solution, the answer is yes. As to why keep on reading.

Let us first explain the process of acquiring the color information from the camera pixel. Each pixel measures light intensity only and to receive the color information cameras usually have an integrated Bayer filter over the sensor. Bayer filter has a 4-grid pattern consisting of two green fields, one red and one blue.

We are naturally expecting to receive all color information (RGB) for each pixel but the individual field is letting only wavelengths for corresponding color (R, G or B) and losing quite a bit of information about the rest of the spectrum. To get the color of each pixel, missing two colors are averaged from the closest neighbors. Meaning that if we wish to read the green value on a position of the blue pixel, we need to average the value from four surrounding green pixels. You can find a more detailed description of this interpolation below.

Now let’s see how this would play out in the real world. Instead of photographing one scene with color and a mono camera, we will take a color image and simulate a Bayer pattern.

The image above shows one-layered RGB simulation as it would look like on a camera sensor without demosaicking, which is explained in the next chapter. Since it is quite difficult to distinguish pixels, you can see a zoomed part with roundabout below.

Debayering

At this point, we have an image as it would come from the single-chip color camera if we didn’t perform any conversion/interpolation on it. To see the picture as an end-user would see it, a process called debayering or demosaicking must be performed. 

The simplest method is to go through every pixel and look at its 8-neighborhood (eight closest pixels) and average the neighboring values for the missing colors. E.g. every blue pixel is missing values for green and red colors and in that case, we would take an average of respectively four green and four red neighbors:

At first glance, there is no difference between the original and interpolated images, but if we take a closer look at the borders, it becomes obvious.

• 

• 

• 

• 

Borders on the interpolated image are weaker than on the original one, which means that they are harder to detect, and their positions have shifted slightly.

Conclusion

To summarize, the process of converting a color RGGB image to a gray one means using partial information (Bayer filter) and trying to achieve the same result as it would be on a mono camera, but with effectively reduced resolution.

If the application requires high precision measurement and there are no advantages in having a color camera, the gray one is a better choice. On the other hand, if you’re not working on a pixel or subpixel level, feel free to use the color camera. In case of high precision requirements with a monochrome camera take care not to buy a pseudo-gray camera containing Bayer filter.

If you have any further questions feel free to contact us at info@subpixel.hr