Week 3 - What I Learned

This is my third week in the Encora Apprentice Program. Even though there are many weeks left, I feel like I have already learned a lot and I am enjoying it.

This week’s main assignment was creating an application to provide user-based recommendations based on huge amounts of data. I learned a lot from it because I had to overcome many difficulties for the first time: first of all, working with lots of data, as my first proposed solutions were throwing out of memory errors and created big files on disk; then working with a different programming language as I’m not well versed in Java but thanks to the previous week’s learnings I don’t have as much fear or uncertainty as before; and processing power because the tests lasted approximately 5 minutes each run. As difficult as it sounds, it was certainly made easier by using Apache’s Mahout library to do the heavy lifting and process the data. Apart from this, another practical experience I had was giving a lightning talk about the TCP protocol: how it works and why it is important; this is the third one I have given, and even though at first I was not very excited to do them, they have allowed me to feel more comfortable speaking English in front of an audience (even if it is a virtual one) and improve my pronunciation a bit.

Related to the Java project, another main topic this week is Test Driven Development, (TDD) which is a software development methodology mainly focused on (not surprisingly) writing tests for every module of the software project so there is a control to supervise changes in a module and make sure they don’t inadvertently break other modules. This methodology, in its essence, sounds simple enough, but as every other methodology there is not just one way of doing things, and this is where things start to get more complicated; the main obstacle comes when trying to test individual software modules, as many modules depend on other ones to function correctly (and test correctly), which kind of defeats the original purpose of TDD: test just one module at a time. There are two popular approaches to this: using Mocks and Stubs. Stubs refer to using the real extra modules to test a specific module, with test data on them; and Mocks refer to using copies or dummy objects specifically written just to comply with the functionality needed to test a specific module.

Moving on to the non-practical part, I learned about Quantum Computing. This is a new type of computing that takes advantage of quantum states to accelerate processing of data exponentially, this can be done by the special property of having multiple states at the same time. For example, whereas a traditional PC uses bits that have a value of 0 or 1, quantum PCs use qubits that can be 0 or 1 essentially at the same time. Even though this concept is not new, in fact it was first proposed around 1980, the technology nowadays is mature and innovative enough to make it feasible; it is not widely used nor easy to implement but certain big companies like Google are already working with these types of PC, even if it is for experimenting purposes as they are very interested in the possible applications of that kind of speed increase. To better understand the speed implications, to brute-force a cryptographic algorithm with a traditional computer would take thousands of years, essentially making it impossible to do, but with quantum computers it can be achieved in a reasonable amount of time.

Also, there were two topics I consider as refreshers but were very knowledge packed: the vim editor and the git version control system. The git version control system is a widely used tool and personally I have been using it in every project since I began writing code but spending time using the tool doesn’t mean I know its workings in and out; in fact, I used the same commands for all my projects and now I have knowledge of many different commands which I am sure will be useful for future projects. In relation to the vim editor, it is a text editor I am vaguely familiar with, which means I know its basic functionality but not much more; but its usefulness is undisputed as it is the default text editor for most of the Linux distributions and learning how to use it (efficiently) is sure to provide many benefits; this objective was fulfilled with the extensive tutorial and list of commands provided by a great article written for Computer Science students.

Finally, there are the non-technical topics, which are just as important as the technical ones (maybe more!). These topics always have one thing in common: they challenge the way I think and how I perceive the world. The most interesting topic for me this week was Moonshot Thinking: this way of thinking indicates that you must aim for 10x improvement in the project you are working on, which will more realistically translate into 2x improvement approximately, but it certainly is much better than originally aiming for 2x and getting 0.2x. Of course, this way of thinking is not applicable to every project you work on, because there are projects where stability and safety is more important; so, this mindset is expected to be added as an extra to apply to certain projects. The other topic is called Inventing on Principle, which implies that creativity is linked to the methodology you are using to create, meaning that your creative process must be adapted to the type of work you are doing (or the other way around) and you must be passionate in what you are doing.

I liked that this week I got to work on some more code, and even though these weeks still feel heavy because of the information overload, they have enabled me to better organize my time, my thoughts and prioritize things.