About 4 years ago I moved to London and began working at a small startup called Goodlord. We had about 14 engineers when I joined. One of the main reasons for joining the team was that in addition to being incredibly talented, they also had a JVM language on the backend. At the time I had never heard of their backend language but I felt the need to gain exposure to the JVM and its ecosystem in order to become a better engineer and as a foray into backend engineering.

The language they were using was Scala. A fairly flexible language at the crossroads between Java and Haskell, that can support both Object Oriented Programming and Functional Programming.

Learning Scala

My main goal at the company was to shift from frontend development to backend development, and in order to do this I was going to learn Scala and show the team how great I would be at it. Although I didn’t know it at the time, one of the criticisms you often hear about Scala is that it is complex, hard to understand, and hard to learn.

Today, I understand why critics think this is the case but I don’t entirely agree. As opposed to some programming languages such as Java which has an ad hoc building block for your every need, Scala has a few very building blocks. The tools which Scala provides are few but they are very powerful and cleverly designed to fit well together. If it is any indication, the latest version of book Programming in Scala has 668 pages and the latest version of Java The Complete Reference is 1248 pages long.

Scala is considered a powerful language. This means that it gives the programmer more control than other languages do, and it allows programmers to express a lot with only a little. An example of a powerful feature is the type level programming that Scala allows at the compiler level. It allows the programmer to prove to the compiler that his program is correct.

I think that learning difficulties related to Scala aren’t necessarily to do with the language itself, but rather the new concepts and use cases the language is useful for. Working on an enterprise Scala program with no prior Functional Programming knowledge is akin to working on an enterprise Java program with no prior Object Oriented Programming knowledge. You would fail to recognise that the code you are seeing contains implementations of formalized and easily identifiable design patterns.

Scala is elegant and stands on its own. Nut without a doubt, a major benefit programmers would get from learning it is the plethora of Functional Programming concepts they would pick up on along the way.

While Scala newcomers are often recommended to begin learning using Functional Programming in Scala (aka. “the red rook”) I don’t think this is a great place to start. The red book is a masterpiece but it is a difficult and academic read. I would instead recommend the less rigorous but still excellent Functional Programming Simplified book, only to then go onto the series of articles The Neophyte’s Guide To Scala, and finally onto Scala With Cats.

Practicing Scala

Scala represents a very small slice of the industry. The StackOverflow 2021 Survey results showed that Scala programmers represent about 3% of the entire industry. All the while JavaScript is the most widely represented at 65% and Java sits at 36%.

Scala in itself can be considered a niche. But even within the Scala community there are two broad use cases which tend to not cross boundaries. There are those using the language for running Spark jobs, and those using the language for backend microservices services.

While I do not have industry experience with Spark, I have read Spark the Definitive Guide. Spark is a framework which is part of the Apache foundation, that helps companies interact with amounts of data which are too big to fit onto a single computer.

Spark is fairly similar to MapReduce, except that it is considered an improvement on the latter because it is significantly faster. It achieves the speed gains over its competitor by mostly working in memory and saving to disk only when its job’s outputs need to be redistributed among the workers, while MapReduce will save each step of its calculations to disk.

Spark’s backend is written in Scala. Today, it provides clients in many different languages. But originally, it had only a few languages to pick from which included Scala. The JVM client is still the fastest and most powerful one to date as it allows access to lower level constructs than its other clients do.

The other niche Scala has found is for backend development. Teams like the ones I worked in at Goodlord and Babylon, sometimes opt for the language for writing their microservices. In virtue of running on the JVM the language has a fantastic runtime to work on with very good tooling, and Scala programmers can also import with Java libraries without difficulty.

The initial enthusiasm for using Scala on the backend had a lot to do with the Lightbend toolkit. Lightbend provides the Akka toolkit which equips programmers with powerful capabilities to create distributed applications. Akka is to Scala engineers vaguely what Spring Boot is to Java engineers. The most famous features it provides are the Actor System, Streams, and HTTP routing functionality. The Actor System is a particularly interesting way of doing concurrency where each actor has its own non-sharable mutable state, and where actors will communicate via messages and queues.

In recent years, programmers have increasingly decided to embrace a purer form of Functional Programming by going with either TypeLevel or Zio. These are competing libraries that expose the data structures and abstractions to have greater control over side effects. Users will be familiar with their respective IO monad, their HTTP routing, and their streaming features. The IO monad is inspired from Haskell and is a fundamentally sound way of controlling side effects in code such as sending requests over the internet, and reading from the file system.

Closing Thoughts

Learning and working in Scala has been an eye opening and rewarding process for me. It has opened my eyes to the realm of functional programming and to the other wonderful languages which exist out there such as Haskell, Lisp, OCaml, Erlang, and Smalltalk.

I truly wished that more of the industry knew about and embraced Scala. But it doesn’t.

Yet, it is pleasing and optimistic to see the trend that Functional Programming languages are becoming mainstream and also that other mainstream languages are borrowing concepts from Functional Programming.

In the future, I will have to make a choice as to whether I want to allow my love of Scala to govern my career although it might narrow my possibilities, or whether I should be more open minded about trying other languages.

I think it is a choice that practitioners of all beloved yet niche technologies have to face. Should I chose to continue with Scala, then I would at least have the comfort of knowing that I am in good company.

Every year Stack Overflow does a Developer Survey where they ask their regular readers, those who might have a minute to spare in particular, to answer a few questions. In 2017 there were a total of 154 questions and over 60000 participants. It is one of the biggest community data-sets out there and luckily for us they published the results.

If you would like to query this data-set for yourself, it is possible. There is an easy to run MySQL project available at https://github.com/AzuObs/stack-overflow-survey.

In this article I’ll be going over some of the key insights gained form an afternoon of querying.

You might need to consider becoming an independent to work remotely.

Fig A Employment Status for non-remotes

Fib A  is the information we have about ALL developers who took part in the survey. The majority of people are full time employees, and there is a big step between those who are working as full time employees and those who have different situations such as being contractors.

Fig B Employment Status for remotes

Fig B contains the information about developers who have declared working remotely “All or almost all the time (I’m full time remote)”. By the way, this will be the way how I decide to categorise remote vs non-remote people throughout this article. We can see that a much higher percentage of people who work remotely do so by working independently rather than being in full time employment.

Fig C Employment Status for non-USA remotes

Fig C shows the trend is even more pronounced if we only consider non-USA based remotes. I assume that is probably because most overseas people who want to work for a US company do so as contractors so as to not require a visa. Whereas most of the US remotes are able to work as full time employees for most companies anywhere in the US. Personally, I’ve worked remotely before, and I indeed had to setup my own company because I was living in France and my employer (Maple Inside) were hiring me from Canada.

Working as an independent might have some downsides. Two that come to mind are that by working remotely you might therefor have less job security, and that independents will incur added responsibilities of running a formal business (company meetings, taxes, and all sorts of paperwork).

Personally, and having worked remotely previously, I would be very confident that these downsides would not outweigh to pros of decoupling my job market from my living place, having more independence, and enjoying more freedom.

Remote workers earn MORE on average their their non-remote counterpart

Average Earnings for non-remotes by country

Average Earnings for remotes by country

These results speak for themselves but could be seen as surprising. Remote workers are often working from rural places with a low cost of living and in globally competitive job market, yet they earn more. For US remote worker this could be because remote jobs attract more mature employees who have a lot of experience and might have chosen the remote lifestyle to offer a better quality of life to their families. For non-US remote workers this could perhaps be explained by those nationals who were able to work at higher rates for US based companies. US companies offering typically better compensation.

Remote workers don’t think remote collaboration is as hard as their office dwelling counterparts

“Is remote collaboration hard?” answer by non-remotes

“Is remote collaboration hard?” answered by remote

Here remotes would much more often disagree with the statement “Remote collaboration is hard”.

Remote collaboration is undoubtedly one of the big challenges remote companies must consider when deciding to hire remote workers. But it would seem that some have found ways to make it work effectively. Nowadays excellent tools such as Slack, Google Hangouts, Google Docs, Trello, and cheap international flights make remote collaboration effortless.

Remote jobs are not only for US residents, although it might be easier for US residents

Work country of all workers

Work country of remote workers

Many remote roles are advertised as US-only. It should often be easier for US companies to hire US residents because the laws would be less awkward to work with than legal systems of entirely different countries. It turns out that out of all survey takers 22% were from the US, whereas 26% of the remote-only survey takers were from the US. So while an observation could be made that remote opportunities are probably more abundant for US citizens, it might not be that significant.

Other insights

The Stack Overflow data-set is full of wonderful eye opening insights and gives the opportunity to answer questions about the remote lifestyle numerically. I may have put forward the above, but there are more insights to be gained. Stack Overflow has made a survey summary of their own where you can find more information https://insights.stackoverflow.com/survey/2017.

Back in 2011 I graduated with a reasonably good Computer Science degree from University of Abertay Dundee. Towards the end of my studies, the financial crisis was in full swing in the UK, and I realized that I although I was technically competent, I wanted to gain experience in a field that would push and develop my business and people skills – universal skills that have wide applications in life. I joined the Oxylane group as a manager, and learnt a lot of valuable life lessons during my years at the company.

Fast forward to 2014 and it had then been a few years since I had seen the inside of a compiler,  although the principles and understanding of computer science were still there and will probably always be with me.

However, my coding skills were rusty and I was going to have to relearn a lot. I took a good look at where the software industry was at before creating a learning plan. Web development was the hot thing at the time, with people coming out of 3 month bootcamps able to find jobs in the Silicon Valley for $100’000.

I didn’t really have the money for a bootcamp, as the ones I looked at costed around $10’000 in tuition and were all almost exclusively in high cost of life areas such as New York and Los Angeles. But as it turns out, one doesn’t need to attend university or bootcamps to get a job in the industry, and I already knew that because I knew quite a lot of software developers didn’t have a degree.

Since I already had the credential of holding a degree, I chose to go the self-taught route and study at my own pace during my spare time.

 

“The research part”

The first part was making sure I wasn’t wasting my time, as my goal was quite clearly to get a foot in the door as a software developer. I spent weeks researching and reading up on the industry and its various entry paths. Luckily, if one wants a career in programming, there are many ways to get one’s foot in the door.

At the time, React existed but was not as widely adopted as it is today. Ruby on Rails was popular but starting to fade. And the MEAN stack was growing in popularity, the Angular ecosystem in particular was revolutionizing frontend development.

The learning plan I came up with included revisiting some important CS fundamentals including Operating System, Networking, Data Structure and Algorithms. And also becoming familiar with the MEAN stack by building a high quality portfolio of one or two polished applications. I still believe that one really good application with interesting design and implementation decisions is going to be more impressive, and a better talking piece during interviews, than building many lesser quality apps.

 

“The learning part”

I learnt from any resource I could find. Of which there are many. My focus was mainly on online courses, tutorials and books.

Keeping my time expectations in check that was by far the hardest aspect of this adventure for me. My mind was fixated on the smashing the “learning plan”, and I wanted it learn it all right there and then! Learning is painfully slow if one is too focused on the day to day progress, it’s much better to take a look at one’s progress after a couple months of learning, because learning takes times.

From my experience and from the others testimonies that I have read, it can take anywhere from 3 to 24 months of learning to land a coding job as a self-taught developer – depending on prior experience and location. I still feel like 12-24 months is the most reasonable target one should set himself: plenty of time to learn things in detail while not going overboard.

One of the pitfalls of learning can be that beyond two years it becomes easy to not be able to see the forest for the trees. There is no end to the amount that you can learn about and you need a cutoff point or will you get lost, and you will lose sight out your outcomes. That is why your learning plan is important: it should clearly set out your goals.

Be careful of “goal creep” however. Goal creep is adding more goals to your original plan that might not add that much value.

It took me 2 years all in all or around 2000 hours. Despite my prior knowledge and degree, it took so long because I wanted to feel very comfortable in my new trade before getting a job and actually be able to make meaningful contributions to my team rather than having to struggle at the beginning.

 

“The Portfolio”

Simply put, you are probably not going to find a job doing something you cannot demonstrate you are already capable of doing.

I made a first good polished application, my Kanban app. Kanbans are a type of time management tool, invented by Japanese car manufacturer Toyota to keep track of the various stages of their projects. If you want to know what a Kanban looks like check out Trello, my app was an almost identical clone that had a “The Simpsons” themed to it.

I also had a second polished application. I made a lightweight version of AngularJS. It is still to this day one of my proudest realizations, and it took 3 months of full time devotion to complete. This is the project that really made me into a better programmer simply because popular open source projects tend to be incredibly well engineered and one because there is a lot to learn from studying and contributing to them.

 

“Outcome”

Because I had a solid learning plan, didn’t try to take shortcuts in learning and didn’t add more goals, I was able to accomplish my goal in the time I had forecasting.

My junior-level portfolio was excellent and once I was ready to start searching, it was the easiest time I had ever had finding a job. All sorts of interesting opportunities came my way.

This experience has perhaps more importantly given me the skills and methodology needed to keep on self-learning throughout my career. It was a huge undertaking and has made me noticeably more confident in myself.

I eventually decided on taking up an Full Stack Developer internship position at TransferWise in Tallinn. After which I had the option to work at the Quebecer startup studio Maple Inside where I currently work as a Full Stack Developer on an ongoing basis.