:: Chasso is turning to the defined direction! ::

Two more days of work and a critical task was done. One more baby step towards the end.

We have now managed to make the Chasso Robot turn to exactly the azimuth/heading we want. That was a very big problem, because if we couldn't achieve it we couldn't command properly the robot. Our main concern was the latency and the different responses that the Lego motors do to the same command.

After the tests and tryouts we manage to create an algorithm for the turn to the correct direction that works like this:

• One motor lowers the base to the floor, lifting the robot on the air;
• The other motor does a rotation of 70% the estimated rotation (we calculated this rotations value based on tests ran by us), with the speed of 50% the power of the motor;
• We then do a loop where we do turns on 20% the power of the motor with cycles of 200 degrees revolutions. This way we can read the compass heading on each cycle and determine if we are on the pretended heading;
• After this we recollect the base, not at 50% the power as we were doing on the descent but at 20%. We discover that doing this we had less chances of the Chasso balance and provoke a change of 2 degrees on the compass heading. This means the robot is again on the floor on exactly the defined azimuth;
• After that we turn the base 70% of the estimate rotation on the opposite direction, again at 50% the power;
• Then we do a cycle using the color sensor the see if the base is in the correct position. If not, we continue rotating the base at 20% the power until that occurs.

We still have some "minor" problems, but they seem very difficult to overcome:

• Changes of heading with less than 10 degrees. The Lego motors are not very good in responding to this small rotations, specially when they carry the weight we use on Chasso;
• The weight of the Chasso makes to down and up base operations not always presenting the same results, does the end of the operation isn't always the expected. We have to ran tests on some dozen times to check if this becomes a problem or not.

The "turn to the heading" operation became more slower, but it's much more accurate (even if the compass only gives even readings - therefore we always turn when we have at least one degree of what we want).

Another good news is the the battery on Chasso wi-fi camera still is above 60%. On the technical specifications we read that for our consumption, it should last at least 10 hours, but it seems it lasts much more than that. We have now about 20 hours after it's initial charging.

:: Chasso and Julio are working simultaneously! ::

Finally we have a breakthrough on a problem that has importunate us for several weeks, the performance issue on having Chasso and Julio robots turned on at the same time. We couldn't have performance to show both videos at the same time and having an acceptable user interface response from the screen controls (in Maria's console App).

After rewriting the algorithm, we took off the timers for image processing and put all tasks regarding the UI on the "Form Shown" event of the main form of the application. Doing this, we maintain a loop that is repeating the update of the user interface, therefore not having any latency during video processing. This is also better in terms of taking advantage of a better CPU. The disadvantage is that the CPU usage is always on a very high limit, but with the video process and live face detection/recognition this is a high resource consuming application. But at least, everything is working and the performance with only one robot at a time is very acceptable, processing at least 2 video frames per second.

On the software front, the next steps are:

• Try to have Chasso to turn the degrees defined by us (not the ones he wants :) );
• Try to have a precise estimation on distance traveled by Chasso, according to the time if moved;
• Implement the full layout of Maria Console.

:: Maria, touch accuracy problems ::

On the touch surface area, there is a zone where the projector reflects and causes a very difficult touch detection, on the remaining area the touch sensitivity is acceptable.

We came with an idea, put another mirror in order to attenuate the intensity of reflected light.
Some temporary changes were made in Maria "body":
1- Added a mirror (a bit bigger than the other) for the extra refletion;
2- Moved the infrared lamps to another position;
3- Removed the tracing paper cover of the IR Lamps;
4- Lifted the camera calibration base.

Thus creating a set of mirror reflections (the projector to a mirror that reflects to another mirror which in turn reflects on the touch surface) that will hopefully mitigate the zone of high light intensity caused by the projector.
But we were not really successful...

That zone, that we refered, is visible at the beginning of the video (0:01), the dark squares should be homogeneous, but it is possible to see the area (bottom below, user perspective) of greater intensity.

Next days will be to improve the changes on Maria body, in order to continue on fine calibration.

:: Global test completed ::

The day began by making with the Switch and Router configuration. As Julio and Chasso are in different ranges of IP's, it was necessary to reconfigure the network in order to make the global tests, because Julio works as an Access Point on 192.168.1.1 and to work with all components at the same time, we needed another network on the 192.168.2.x, so that Maria console can comunicate with both networks (one with cable and another with wi-fi) and to maintain the necessary internet connection :)

For the first time we had everything to to a test with all the components we are going to use on the final presentation. On the Chasso robot adjustments were made to balance the weight when doing a lift. With the new battery, it was necessary to do tests to find de adequate place to put it, because we need the balance to be stable, even when Chasso rotates. Everything ran smoothly. This operation was successful.

Regarding the new battery, we were very happy. We were not sure about the battery life, but after 1:30 the battery remains above 80%. Another successful test (and another good ebay buy). Then came the big test, try to connect simultaneously Julio and Chasso to test whether the two could be ran together, specially the video update. After some hesitation, everything went well. One negative test point has what we were expecting, was the performance on UI update. We have not enough computational resources for the processing of two videos. The two images are captured, but the image processing is not done in an aceptable time. We have to improve this section by optimizing the code and try new ways of processing the video.


In relation to Maria, we are having some trouble (girls!). The driver used is not working as expected. We have to research more on how to optimize this also. In summary, today was very productive day, but solving some issues, brought us another. The fight continues...

:: First global test is near ::

Today arrived our last purchase for the system, the battery for the wi-fi camera to mount on top of the Chasso Robot.

After our tests with simulated weight and volume, we ordered the TeckNet AE-iEP387-7000mAh 5V 2A because it seems like te best choice.

If the Chasso behaves ok with the weight and we manage to get a position for it that doesn't not afect the balance, we can finaly do our first global test, with all components working simultataneasly. Our main concern is the performance of everything (according to our previous tests). More news tomorrow...

:: WinForms still rules (in some situations) ::

After being all week working on code translation from WPF to WinForms we got together one more day to do some tests and evaluation.

So the agenda for the meeting was:

1. Test the WinForm App with the Chasso robot;

2. Test the WinForm App with Julio robot;

3. Test the WinForm App with both robots;

4. Test Julio with a iPhone on top, sending GPS coordinates;

5. Test traveling salesman algorithm;

6. Define intermediary report content;

7. Discuss implementation of Maria's UI;

8. Test the MacBook Pro connect to Maria.

We managed to do a lot of work on this day:

1. We had changed the Maria's Console application from WPF to Windows Forms because we were having a very bad performance. With both cameras and sensors from both robots connected, we couldn't get from than 1 frame per second on video (even with only one working at a time).

The tests with only the Chasso Robot and in WinForms, even with more poolings from sensors per second are much better than on WPF. The video frame passed from 1-2 frames per second to 5-8.

2. Once more we run into some difficulties. Because almost all examples from .NET ARDrone SDK Wrapper are for WPF we had to rewrite a lot of code. Until now we couldn't finish this task. News will come on the following days.

3. As on point 2, we will have to wait for the programming to be finished so that we can test it.

4. This was the fun part of the day. For the first time we took the ARDrone (Julio) for exterior flight, but not before we did some interior tests with an iPhone on top of it. By the end of the day we had some conclusions and changes to the project (a part from one more big incident where we tread the hull once more):

• The iPhone is very heavy for the Julio. It's 140 grams are too much for it. To make it carry that weight and have a stable flight, we shouldn't use the protection hull and tweak the Drone, but that would make us loose the warranty. We also confirmed this on web forums;
• The iPhone wi-fi network is always trying to connect to the local available networks, making the iPhone App become more unstable and affecting the flight (Julio does less responses of commands per second); 
• The Android based Z71 Boston mobile phone were more lightweight (around 100 grams) and it's wi-fi network didn't affected the ARDrone as much as the iPhone;
• Based on this, we will change the project and use the Android Phone has our GPS on Julio. We will now look form information on our to program this on Android.

5. We left this item for a future meeting.

6. We discussed what we should prepare for this report, that is due to be delivered on the 15th of May.

7. We did some small tests in WinForms, but we need to get back to this on the following days.

8. Done with success.

:: Chasso Navigation and Traveling Salesman Algorithms done ::

During these days, because classes have already started we didn't got the time to get together, therefore we've been advancing on tasks we can do by ourselves.

We have now finished an implementation in C# of a class to solve the "Traveling salesman problem" based on the book "A java library of graph algorithms and optimization" by Hang T. Lau. We will use this as an agent in Artificial Intelligence for discovering the fastest way between 3 points and return "home".

We also designed a flowchart with a new algorithm for the Chasso navigation. With the changes made we hope to circumvent the performance and latency problems we were having. Now we are going to implement it and do some tests. Probably more news on this will be available next weekend.

:: Maria's UI ::

Yesterday we were very disappointed after we detect some problems, mainly performance issues that can derail the whole project, but we have only one way ... to move forward.


On the other side, we have reached the base of the final version of the Maria's UI prototype. It is subject to change, according to eventual implementation problems, but shouldn't be very different from this.

:: Serious Problems ::

After a promising week, this weekend we met again for a full day working together with a tight agenda:

1. Continue and try to finalize Maria's UI prototype;

2. Do some more tests with different materials on top of Maria glass;

3. Continue the programming of the "Travelling salesman" algorithm;

4. Implement new wi-fi camera for Chasso, together with Chasso navigation (this one using all the changes made to it on the last days, mainly the color sensor to know the whereabouts of the base).

If on points 1 to 3 of the agenda, we did some steps forward. Not finishing it, but came close to have it done or at least, got more near the end, in the point 4 of the agenda we found a major problem that could become a major impediment on the implementation of our concept... the Lego NXT brick has a very high latency between the sensors readings (we have to wait more than 500 milliseconds) for a complete reading on all 4 sensors. This by itself we already knew, the problem is when we had the time consumed for processing the video for the wi-fi camera, the time do detect faces on that video and the other tasks, the system becomes very, very slow. It's impossible to work only the Chasso, so using both Chasso and Julio at the same time, seems even more impossible.

 
We are using a MacBook Pro with an i5 and 8 Gb of RAM and the processor is getting to around 40% of use, so what we will do to try to solve this issue is to break some of the processor consuming tasks in threads, so that we can use multiple cores of the CPU and get a little more performance. News on this on the following days...

:: More changes on Chasso ... and Marias's UI prototype is looking good ::

One more day of school and work holidays, one more day of full work on the project.

Today, after some fine tests, calibration and fine tuning on Maria, we got the conclusion that will be hard to  have the PS3 camera to capture all the image projected, therefore we should leave a margin around the working area, because even with a full project image, we can't capture with precision the touch inside that margin.

But most time and work were spent changing the Chasso Robot. With the initial tests done today we came to the conclusion that with the wi-fi camera and it's battery on top of the robot created several problems:

• The weight is too much on front, most of the times when Chasso goes up to turn, it created a balance that made the robot touch the ground and therefore not turning the degrees we wanted;
• The weight is too much for the rear axis. The axis folds with it, affecting the direction;
• Because of the weight, if the robot goes up and turns more than 90 degrees it becomes very unstable and touches the ground;
• Also because of the weight the base doesn't hold to ground as easily and it becomes slippery, so the turns are not done with the desired turn degrees.

With all this problems, we had to put our hands on it and do a lot of changes:

• We changed the position of the wi-fi camera. We had to put it a little bit more to the rear;
• The compass was also sent to the back, because of all the motors weight, Chasso was bending forward, so we need to shift more weight to the back;
• We took the rear bumper off, was well as the wi-fi battery compartment that we previously assembled to loose some more weight;
• We put 2 elastics on the base so that we could have more adherence to the floor;
• We managed to get the rear axis more to the back. This way we could put an axis piece that goes from one wheel to the other and a support for helping not to bend. This helps the Chasso to move forward with less deviation of the planned direction;
• The battery of the wi-camera has to be transported on the back, otherwise, sometimes  Chasso will bend forward when he goes up (but this had to be retested with the final battery, because that one has not arrived we had to used a mockup one);
• And the final and more dramatic change, we added the color sensor. To do this, we had to prescind of one of the touch sensors (the left bumper one), because the Lego NXT brick only has 4 entrances for sensors (ultrasonic, right bumper, compass and now color). We also changed the color of one of the base pieces, for an orange one. With this we can can know if the base is on the correct position to go up and do the turn, or if we need to turn the base to the correct position before going up.

  

:: A very productive day in all fronts ::

Today, being almost all of us on holidays from work (and university) we manage to met and work on several of the components and we got some good results and advances.

Maria

We went to look for a new type of frosted glass, because until now we were having problems with all the materials we've been using (glass, paper, plastics, etc.). We need some transparency so that the image can be seen, but also some degree of opaqueness so that the infra-red and the lamp light are blocked. Also, the surface on the bottom can't be reflective or the readings on the camera are very bad.

After several hours and glass shops we managed to buy on for around € 20.00 that it seemed to respond to our needs.

After some calibration and tests we got the feeling that this was the final one. This will be our surface of choice, but probably will even put a plastic on top to block more the infra-red light, because after some time this light produces some fatigue to the eyes. Further tests will determine this.

During the calibration and balance tests we also noticed that with this glass the infrared light wasn't being dispersed with homogeneity on all the glass surface. We thought on different solutions, but applied a very simple one used on flash photography, we just cover the infra-red lamps with tracing paper. It worked!

We gave a huge step on putting the surface table working as pretended. Next days will be spent doing tests, calibration and fine tuning.

We also continue working on the prototype of Maria's software user interface. This is proving a huge task. We have a lot of components on screen and we want to use has much interaction has we can, so putting everything to work and having a futuristic, but reliable and easy to use interface takes some work.

Julio

One more day on autonomous flight tests. We have now put it to work on the 3 axis and having a smooth flight, but there we continue with some problems:

• We can only process a maximum of 2 video frames per second, which is too long in terms of flight navigation. The delay between navigation commands becomes very high for having a more smooth "follow the face";
• The battery doesn't last more than 1 hour of tests (12 min flying) and because it takes 1h30m to charge and we only have two, we have to stop a lot.

Nevertheless, this was a good day, Julio is now following the "locked" face. We still have to solve this issues, but we think it can be done.

Chasso

On this front we also had some news today. The wi-fi camera to mount on top of Chasso arrived.

We did some initial tests to check if you could use it's pan and tilt functions which were successful, therefore we can now included this functionality on the Chasso.

We also did some tests on the maneuvering because of the weight added and it became obvious that this additional burden was a problem. We can solve it by distributing more weight on the back (we will test this later on with the batteries for the camera).

On general this was a very good day, we managed to move some "yellow stickers" from one phase to the another :)

:: Report of the last two days - Maria's UI prototype and Julio autonomous flight ::

Today, we got together for 3 more hours to try to solve some loose ends and to discuss the work we've been doing individually on this last two days and what next steps we should do.

The biggest discussion was around Maria's User Interface prototype. We already have the most difficult part done, but the details are killing us. We took more than 2 hours of the meeting just discussion icons, way of operation and rearranging the screen. Let's see if until the weekend we can have a "closed" prototype we can start implementing on.

During this days, one of the advances we did was on the implementation of the algorithms to load, identify and recognize the faces on Julio video, with the locking mechanism that enables the autonomous flight.

Unfortunately our main obstacle during this days was not the programming and implementation of the algorithms for face recognition and identification, but identifying the flights distances and speeds continuous to be a very hard and time consuming task. This was so bad that on one of the autonomous tests flights Julio just made a crazy turn and went upside down against a lamp, braking it's protection hull. 

:: More Maria's UI Prototype ::

We decided to do a lot of changes since last meeting. Also we have readjust the UI to receive Chasso and Julio controls. 

So we decided:
- To rearrange the layout;
- Ensure that the layout is more intuitive;
- Keep the "Start/Off" buttons far from the application controls, to avoid unintentional mistakes;
- Add some new buttons, since we must interact with the map itself;
- Remove the controls of Julio, since they are not required because we have the joystick;
- Add a knob button to interact with  Chasso, with information on the turning degrees;
- Introduce some info of the sensors with relevant information.

:: Maria's UI Prototype ::

One more Sunday working on the layout of "Maria", using different designs to achieve the version that most pleases us.

but still not happy ... there are some things in the middle section that must be changed.

:: Bad news on Maria, good news on Chasso ::

Saturday is the only day of the week we can be together all day long, so this is day were in do our biggest or longer tasks.

Today, we left the morning for find a new surface for Maria (surface table). Although we need a surface that it's non reflective, at the same time, the surface must have some level of transparency, because the image must pass to the users. So, we already did tests with different types of glasses, plexiglas, different type of papers and different type of plastics , but unfortunately we continue with some problems, so this morning, we went to different shops to look for materials we could use. 

After all the browsing around, we shopped for one architect paper and some plastic, but once more, we didn't had luck. We had problems again with air between the glass and the paper and the plastic was too transparent. The search for materials will continue on the next days.

The other major task for the day was trying to finalize the 1st robot (Chasso) and start doing some movement tests. It seems we have a winner... version 4 feels like the final one.

After some quick tests we manage to put Chasso to work almost on a straight path and turning on an angle of 90 degrees. On the next days, we will do some tests to get familiar with ratio of speed/distances versus strength applied to the motors and angle of turning versus rotation applied to the motor.

:: Autonomous flight with Julio - locking on the person is being looked for ::

One of the functions we're implementing on Julio is the ability for after recognizing a face, if the user want he can give a command to Julio to "lock" on that person. Doing that, Julio will go to an autonomous flight mode (like auto-pilot) where he follows the face of the person and according to the face position on the screen it positions itselfs, for example:

• Face center is on the left of the screen - Julio goes right on X axis (roll);
• Face center is the upper part of the screen - Julio goes up on Y axis (gaz);
• Face area is smaller then a minimum amount - Julio goes forward on Z axis (pitch).

Here is the flowchart diagram of the algorithm we are implementing:

All day was spent on this algorithm and transforming it from paper to programming and then adapting, because Julio auto-adjusts itself according to the flight conditions (wind, height, etc.) so in reallity it's not hoovering exactly on the same place as you "left it" before. But, one of the major factors for more working not being done is the batteries. Julio came with 2 batteries, but even that is not sufficient when you spent all day working with them, because they do not charge as fast as you need. Maybe we have to go to ebay once more :(

:: Chasso V4 ::

News from the postman. We received the piece we had ordered on ebay and moved immediately to action, the construction of Chasso version 4.

We are confident this will be the final version (or be near it). This version uses one motor for locomotion, one for lowering a base and another motor for doing the rotation we want.

This version is based on a version created on a book Extreme NXT: Extending the LEGO MINDSTORMS NXT to the Next Level, 2nd edition, by Michael Gasperi and Philippe E. Hurbain (link here), which was built for an Odometer, exactly what we are doing.

We then need some adaptions for carrying bumpers, ultrasonic sensor, compass and web camera.

A big thanks to Nelson Russa who let us is Lego RCX so that we could have enough lego pieces to construct what we needed.

Next weekend we will do the tests and get back to blog with the results.