An Innovative AI-powered Computer Vision And Gesture Recognition System

How does the brain interpret what we see and how can computers be made to mimic the natural workings of the human brain when it comes to sight? That is the question that computer vision technologies seek to answer. Today, many technologies use computer vision in artificial intelligence, or AI. This AI rely on neural networks and they have to process a large amount of data in a very short space of time. Many AI-powered computer vision systems have been introduced into the market and they are being used in hi-precision surgical robots, as health monitoring equipment and in gaming systems. Heard of the Google computer vision, or Google cloud vision API? Those are examples. But engineers want to go beyond these computer vision applications. They want the AI-powered computer systems to recognize human gestures so as to complement its visual capabilities. That is why gesture recognition technology has become a hot topic in computer vision and pattern recognition.

 

The drive to create AI systems that recognize hand gestures came from the need to develop computer systems and devices that can help people who communicate using sign language. Early systems tried to use neural networks that incorporate the ability to classify signs from images captured from smartphone cameras while this data is converted from pictures to text. They were systems that involved computer vision with image processing. But AI systems have grown more advanced and more precise than those humble beginnings. Today, many systems seek to improve on this visual-only AI recognition system by integrating input from wearable sensors. This approach is known as data fusion.

Data fusion is the process of integrating more data sources into computer systems that make these systems more reliable and accurate than if the data was coming from a single source. AI-powered computer vision systems incorporate date fusion using wearable sensors that recreates the skin’s sensory ability, especially the somatosensory functionality of the skin. This has resulted in the ability of computer systems to recognize a wide variety of objects in their environment and increase their functionality and usefulness. But there are still challenges which hamper the precision and the growth of these data. One of these challenges is that the quality of data from wearable sensors are low and this is as a result of the fact that wearable sensors that have been produced are bulky and sometimes have poor contact with the user. Also when objects are visually blocked or there is poor lighting, the ability of these AI-powered systems are reduced. One area that has been troubling to engineers is how to efficiently merge the data coming from the visual and the sensory signals. Therefore, this has led to information that is inefficient, resulting in slower response times for gesture recognition systems.

In an innovative approach that is said to solve many of these challenges, a team of researchers at the Nanyang Technological University, Singapore (NTU, Singapore), have created an AI data fusion system that drew its inspiration from nature. This system uses skin-like stretchable sensors made from single-walled carbon nanotubes. This is an AI approach that closely mimics the way the skin’s signals and human vision are handled together in the brain.

How the NTU artificial intelligence gesture recognition system works

The NTU bio-inspired AI system was based on the combination of three neural network approaches. The three neural networks that were combined are: 1. a convolutional neural network which is an early method for visual processing, 2. a multi-layer neural network which was used for early somatosensory information processing, and 3. A sparse neural network which fuses the visual and the somatosensory information together.

Therefore combining these three neural networks makes it possible for the gesture recognition system to more accurately process visual and somatosensory information more efficiently than existing systems.

The lead author of the study, Professor Chen Xiaodong, from the school of Material Science and Engineering at NTU says that the system is unique because it drew its inspiration from nature and tries to mimic the somatosensory–visual fusion hierarchy which is already existing in the human brain. According to him, no other system in the gesture recognition field has undertaken this approach.

What makes this system particularly accurate in data collection is the fact that the stretchable skin sensors used by the researchers attach comfortably to the skin and this makes the data collection process not only more accurate but makes it to deliver a higher-quality signal which is vital for hi-precision recognition systems.

The researchers have published their study in the scientific journal “Natural Electronics”.

High accuracy even in poor environmental conditions

As a proof of concept the bio-inspired AI system was tested using a robot that was controlled through hand gestures and then the robot was guided through a maze. It was discovered that the AI system was able to guide the robot through the maze with zero errors, compared to the six recognition errors from another visual recognition system. It then seems evident that this bio-inspired AI system is more accurate and efficient.

Also it was tested under noise and unfavorable lighting conditions. Even under this unfavorable conditions the bio-inspired AI system still maintained its high accuracy. When it was tested in the dark, it worked efficiently with a recognition accuracy of over 96.7%.

The authors of this study said that the success of their bio-inspired AI system lies in its ability to interact with and complement at an early stage the visual and somatosensory information it was receiving even before any complex interpretation is carried out. This makes it possible for the system to rationally collect coherent information with low data redundancy and low perceptual ambiguity with better accuracy.

Promise of better things to come

This innovative study shows a promise of the future. It helps us to see that humans are one step closer to a world where we could efficiently control our environment through a gesture. Applications that could be built for such a technology are endless, and it promises to create a vast amount of opportunities in Industry. Some examples include a remote robot control over smart workplaces along with the ability to produce exoskeletons for those who are elderly.

The NTU team are aiming to use their system to build virtual reality (VR) and augmented reality (AR) systems. This is because their system is more efficiently used in areas where hi-precision recognition control is required such as in the entertainment and gaming Industries.

Material for this post was taken from a press release by the Nanyang Technological University, Singapore.

Python List And Sequence Comparisons And Sorting Based On Lexicographic Orders

According to the documentation, comparing sequences is done based on lexicographic order. That is just a way of saying that comparisons between sequences are done based on their position in dictionary order if alphabets, and if they are integers, based on their position in the number line. Comparisons could be done using the lesser than operator, <, the greater than, >, operator, or the equal to, ==, operator. It really gets interesting when you are dealing with sequences that have a mix of both alphabets and numbers. These comparisons and many other comparisons are what we will be discussing in this post. We will also show that the python list sort method and python sorted function are based on comparisons.

 

Note that these comparisons are Booleans. That means, they give you True or False when these items are compared.

Let us compare two lists in python and see how the comparison works on sequences. When objects are to be compared, they must be of the same type. If they are of different types, python will return a TypeError.

1. When the two python sequences are of the same length and type.

The code above compares n and m which are python sequences of numbers. You can see that they both only differ in their last items in the python list. I just used this example to show you that when python compares two sequences of the same type each index is compared to the corresponding index until a mismatch is found, and then based on the lexicographic order, one could be found to be greater than, lesser than, or equal to the other. In the code above, n was lesser than m because index 2 in n, which is 3, is lesser than index 2 in m, which is 4. Indices start from 0.

2. When the two python sequences contain items of different types

When the two sequences being compared have items of different types, python will return a TypeError. Note the code below.

When run, the above code returns a TypeError because string and integer types cannot be compared.

3. When the two sequences are the same length and contain the same items of the same type.

When you run the code, you would realize that they compare equal. What python does is walk through each item and compare them index to index. In this case, all the items compare equal. But what if one list is shorter than the other and all the items compare equal. What does python decide? See the code below.

When the code above is run, you would see that python takes the shorter of the two sequences as the lesser one when they compare equal, index for index. It now uses the len function to decide on the equality or non-equality of the two sequences.

I have used python lists in these examples, but you can use any sequence like a python string, tuple, or range.

Comparison of user defined objects

Can we take this notion of comparison to user defined objects? Yes, of course. You can provided your user-defined object has the appropriate comparison method. Or in other words, provided it implements the __lt__(), __gt__(), or __eq__() special methods. If that is the case, you are good to go. Here is an example of how comparison could be done on user defined objects.

When you run the code above, you can see that objects of the Length class can compare themselves even though they are not sequences.

This ability to overload native methods and python operators gives a programmer great power. That power comes with enormous responsibility. One of such power is the ability to use the concept of comparisons to carry out sorting. Python has two functions to do that, and they are the built-in python sorted function and the list.sort function that comes with the python list class. These two functions work based on the concept of comparison to sort items in sequences. We would be using the built-in sorted function since it is generic.

The python sorted function

The sorted function creates a new sorted list from any iterable. By default, it sorts based on lexicographic order and in ascending fashion. Take the following code for example.

When you run it, it sorts the list of fruits in dictionary or lexicographic order. The underlying mechanism at work is a comparison of each of the fruit items. That is why you could change the order of the sort. The sorted function has a reverse keyword argument that you can use to do that. By default, reverse is False but you can switch it to True to sort in reverse lexicographic order. Let’s do it.

After running the above, you can see that when I set the reverse argument to True, it sorted the items in the fruits list in reverse order.

There is also another keyword argument that is useful when you have items in a tuple or a nested list and you want to specify which order to sort the items. For example, if we have a list of tuples comprising names and ages, how do we sort the list such that the ages takes more prominence in the sorting order before the names? This is best defined using the key keyword argument in the sorted function. In the code below, I would use a lambda function to specify what the key should be. Lambda functions are anonymous functions. The lambda function will sort or compare the items in the python list based on their ages.

As you can see, ‘David’ who is 20 years old comes first in the list, followed by ‘Rose’ who is 25, then by the two other students, ‘Michael’ and ‘Daniel’ who are both 32. But there is a problem with the sorting. The sorting is not yet complete. If Daniel and Michael are both 32 and compare equal for ages, then naturally we should expect Daniel to come before Michael in the sorted list. That’s right. So, let’s add one more power to our key. This time, we would tell the key argument to first compare by age, and if ages are equal, to compare by names. The code below shows how it is done. The only difference from the above code is that I added x[0] to the statement in the lambda function and that makes it possible because for each item in the list, x[0] is for names while x[1] is for age. To make them compare equal, I then cast the key for age to a string.

Here is the code.

We now have a well sorted list where ‘Daniel’ comes before ‘Michael’.

Let’s take this a bit further and give more power to sort any object, not just custom data structures like sequences. We could extend this power to our custom Length class that we described earlier. Let us be able to sort any sequence that has Length objects.

This is somewhat simple because I have already given Length objects the power to compare themselves. Remember, sorting depends on comparison. So, having this power, we can do sorting on length objects.

The only functions added to the code above for the Length class is the __str__() special method. This gives us the ability to print out the values of the objects, as well as the sorted function.

So, I encourage you to use this power with responsibility. Python gives you lots of power to do all sorts of things with your objects, even to compare and sort to your desire.

Do Face Masks Really Protect Against Covid-19 As Claimed?

Public health officials have launched a protracted campaign to make us believe that wearing a face mask prevents the spread of Covid-19, the latest pandemic. But is this true? That was the question on the mind of a Duke physician, Eric Westman, who was a champion for people putting on masks. He wanted to be sure that he was recommending the right prevention technique to people and businesses. So he decided to carry out a proof-of-concept study. A proof-of-concept study is a study that aims to test whether a technique or method is really as effective as claimed. Also it is used in science as a testing phase to verify the feasibility of a method, a technique or even an idea. When scientist carry out an investigation in science, they start with an idea like this.

 

Doctor Westman was working with a non-profit and he needed to provide face masks to people. He was also skeptical about the claims that mask providers were making about how effective their marks were against a pandemic like covid-19, so he went to a chemist and physicist at the University, Martin Fischer, Ph.D. and asked him to carry out a test on various face masks. The test they did was based on both surgical masks used in medical settings and cloth face masks. They also carried out tests on bandanas and neck fleeces used by people who claim they can prevent the spread of covid-19.

Fischer’s line of work usually involves exploring the mechanisms involved in optical contrast while doing molecular imaging studies. He was intrigued by the doctor’s challenge so he set out to help him. For the study he used materials that were freely available; something that can easily be bought online. These materials include a box, a laser, a lens, and a cell phone camera.

From the study it was reported that it proved positive and showed that face mask were effective at preventing the spread of Covid-19. They recently published their scientific study article in the journal “Science Advances”. Despite this being a low cost technique, it helped to prove that face mask prevent droplets coming out of the mouth while speaking, sneezing or coughing from being transmitted from one person to another. They reported that while carrying out the study they could see that when people speak to each other, cough, or sneeze that molecules of droplets are passed from one person to the other. They also confirmed that it is not all masks that are effective at preventing the spread of droplets. Some face coverings were seen to perform better than others in preventing the spread of droplets.

So how did the masks compare? They tried out the proof-of-concept study on various masks and compared their effectiveness. They found that the best masks were the N95 masks which were used in medical settings. They also found that surgical masks and masks made of polypropylene were equally effective in preventing the spread of droplet molecules. Face masks which were made from cotton allowed some molecules to pass through but were found to have good coverage. They could eliminate several droplets that were being passed when people were speaking. Overall it was shown that bandanas and neck fleeces should not be used at all or recommended as face covering. This was because they were ineffective in blocking the spread of droplet molecules.

When the physicist was asked if this was the final word on the subject, he replied in the negative. Therefore more studies need to be carried out because this is just a demonstration to show the effectiveness of various face masks. The study was done to help businesses see that they can carry out these tests themselves before investing on any type of face mask or face covering.

When asked on the benefits of the study, Westman, who was inspired to start it, said that many people and businesses have been asking him about how they could test all these face masks that were new in the market. So he decided to show that businesses could carry out the tests themselves with very simple materials. He said that the parts for the testing were easily purchased online and they were putting out this information to help others.

As they hoped, they have shown that various face coverings that were being promoted by public health officials were indeed effective in preventing the transmission of molecular droplets from one person to the other.

Although this is just a proof-of-concept and not a rigorous testing technique, one can confidently recommend the use of face masks to individuals and businesses because they really work in preventing the spread of covid-19. My advice to everyone is to stay safe, wear a face mask, and help stop the spread of covid-19. We will see an end to this soonest. Please, carry out social distancing and regular hand washing to prevent the spread of this current pandemic.

Material for this post was provided by the dukehealth.org website.

Application Of The Built-in Python Enumerate Function

 

How many times have you ever wanted to loop through a list or tuple while keeping count of the number of times and you ended up doing it with a for-loop? For beginners, I think the answer would be many times. But that is not pythonic. For example, I notice this code often among many python programmers who are not aware of the existence of the built-in python enumerate function. They keep an antipattern consisting of a range over the length of a list while keeping a running total of how many times they have gone through the range object created.

fruits = ['mango', 'pawpaw', 'lettuce', 'orange', 'banana']
for i in range(len(fruits)):
    print(i, fruits[i])

Please, if you are doing this, it is harmful to your code and not pythonic because your code is not easy to read and is vulnerable to your making typing errors.

To prevent code like this, python has a nice function that does it elegantly called the python enumerate function.

The python enumerate function

The syntax of the python enumerate function is enumerate(iterable, start=0). This is a function that accepts an iterable as first positional argument with a start value for the counter to the iterable. The default for the start value is 0 but you can specify a starting value of your choice. When you enumerate an iterable, it gives you an enumerate object. The benefit of such an enumerate object is that it is an iterator with which you can use a for loop to return a count of the items in the enumerate object. We will get to that later. But let’s see how it works with an example of enumerate.

If you run the above code, you will find that the python enumerate function gives an enumerate object. Now, let’s loop over the enumerate object.

In the above code, I used the default start of 0 and you can see that when the counter was printed, the counter started from 0. We can tweak that feature to get a starting counter of our choice. Now for some examples of enumerate function.

So, you can see how powerful this little known feature in the python programming language is. The enumerate function gives all iterables an advantage the python dictionaries already have, which is an index notation that is compact and reliable.

So, the next question is: what can we do with the python enumerate function?

The simple answer is a lot.

Application of the python enumerate function

1. Add a counter to a list or iterable
.

Just as in the example I gave above, there are lots of times we want to add a counter to a list or tuple, especially if it is a large list or tuple. This handy function makes it possible. You can use the counter as a key to get an item in the iterator that you want to use. For example, you have an extremely long list of cars and you want to be able to know them by number. You could use enumerate to give them a count value starting from any number and then retrieve each car based on the counter.

For some example:

What if we wanted to know the first, or third car?

2. Convert to a list or tuple
.

We could convert the python enumerate object which is an iterator to a list or a tuple and use the handy functions of python lists or tuples. It is so easy to do. Just use the enumerate object as the argument to the list or tuple and it is easily done. When you have the enumerate object as a list, you can then use python list functions on them instantly.

Here are some code examples.

You can see from the above that I used the index of the cars_list list to get the last item in the list of cars, and then used the len function of the list to find out the number of cars in the list.

You can read about the rationale for the enumerate function from PEP 279 and its documentation at python.org.