To sleep or not to sleep? That is (not) the question

04 Aug 2021

Fitbit? More like Sleepbit

My amazing girlfriend once had a great idea for a Christmas gift - a small Fitbit. She also got one, which helped us stay fit by encouraging each other to beat each other’s stats. A great side effect I discovered only later on is that wearing the Fitbit every day for 1.5 years generated tons of detailed sleep tracking data. And I really mean tons.

The next thing was to get all that data. Knowing how companies tend to make their products as limited as possible in terms of access to your own data, I didn’t have much hope. However, it turns out that Fitbit allows you to export all of your data in… JSON and CSV formats! What a news was that to me!

Exploration time!

To uncover what my sleep data tries to tell me, I did an in-depth analysis of 1.5 years’ worth of data. An interactive Jupyter notebook is available here.

Main conclusions

My analysis:

Seems to confirm that going to sleep on time improves sleep quality.
Confirms my feeling of “not being a dreamer” - I regularly get less REM sleep than I should for my age. This will have to be further diagnosed by a sleep doctor.
Confirms that I’m a “light sleeper” quite literally - the vast majority of my sleep stages are light sleep, sometimes to the point of being disproportionately frequent. This partially explains my occasional tiredness during the day.
Shows that the duration of a single deep sleep episode as well as the total deep sleep time per night is highly variable, which helps explain days where I feel very refreshed one day and tired the next one.

Overall, this was a great exercise in data wrangling and using using Plotly to make interactive graphs. I hope you enjoy it as much as I did!

On speed limits and lane counts on highways

16 Jun 2021

I’ve recently moved out of Paris due to soon ending my wonderful journey with Datadog. Thanks to my parents helping me out, it was much easier than doing it all alone. Still, this was a much more logistically complicated process than I initially thought:

I couldn’t travel by airplane, as the amount of luggage to transport would cost me an arm and a leg. I didn’t want to leave my luggage to be transported by a moving company, as I had a few very important personal items.
COVID is still a thing, meaning - tests. Coordinating taking a test within 72h of arrival to France (travelling through Germany) required a bit of planning

Needless to say, while driving / riding a car for > 1700 km one-way (Warsaw <-> Paris), you have lots of time to think. And think I did. “What about”, you ask? Naturally, about highways, which constituted probably > 75% of the total distance travelled by us.

A couple of my observations:

Significant parts of German highways (“autobahns”) have no speed limits, no strings attached. No such thing in Poland and France.
Where speed limits exist, there are noticeable differences between countries (Poland having the highest limit of 140 km/h, France and Germany both having 130 km / h)
The distribution of speed limit areas changes between countries (it seems there are more in Germany than in Poland)
Polish drivers seem to obey speed limits more rarely than German and French drivers (purely based on my observations - perhaps this is because Poles are rarely fined due to many fewer speed cameras, and the fines being lower?)

Given all the above, you’d think that getting from point A to B in Poland would be the fastest of the three countries. Turns out, you would be wrong. After nearly 30 h spent on highways in all these countries, here are my observations and hypotheses that explain this apparent paradox:

German and French highways primarily have 3 lanes, while Polish ones have 2. This is a surprisingly big deal, much more important than any speed limits. Why? Highways are the primary “arteries” through which heavy trucks transport goods. These trucks are slow compared to passenger cars, and so they stick to driving on the rightmost lane. Here’s what happens depending on the number of lanes:
- On a 3-lane highway, this leaves the middle lane for medium-speed passenger cars, and leftmost lane for high-speed cars and overtakers
- On a 2-lane highway, the only lane left for passenger cars (both medium and high-speed) is the leftmost one.

Now, what happens if a truck decides to overtake another truck, or if a particularly slow passenger car stays on the left lane? On a 3-lane we’re fine. This still leaves the leftmost lane free. On a two-way, we have a problem. All the cars on that lane will be speed-limited by the slowest-moving car, with no way of overtaking.

This phenomenon was extremely obvious when 3-lane highways in Germany temporarily narrowed to 2 lanes - the whole traffic slowed down by ~ 20 - 30 km / h on average, regardless of the speed limit. This seems to me like the biggest factor behind why driving on Polish highways will be slower than on German ones, despite a higher speed limit.

While German speed limits seem more frequent than Polish ones, they are introduced gradually, while the Polish ones - more abruptly. E.g. if there is a sharp turn, the speed on a German highway will be gradually reduced to 120 -> 100 -> 80 or similar. In Poland, there is typically only one speed limit reduction, right down to the lowest speed. This makes “fast traffic jams” on the Polish highways, which contributes to slowing down the overall traffic.
German highways are famously smooth. This means that no speed reductions are needed before surface irregularities such as holes.

It’s clear there was a lot of clever planning done when designing these highways in Germany, and this really contributes to making it a pleasure to drive on them.

A quantitative look at happiness

30 May 2021

Jupyter notebook link

Data to the rescue

Naturally, to have a go at answering these, I had to find relevant data. The World Happiness Report data looked great for my purpose. I also found a suicide rates overview dataset which I later merged with the happiness one. Data cleaning clearly took the longest due to some inconsistencies and missing data in several datasets. However, afterwards, it was mostly smooth sailing.

The complete analysis of both datasets is here - have a look if you want to dive deeper into the methods used and see the graphs.

Conclusions

All in all, the data mostly supported what I learned in the course, and gave me some answers to my questions. The most important conclusions are:

1. Money is not as important as we think in making us happy

2. Good relationships and good health are important contributors to our happiness

3. The degree to which you’re happy / likely to commit a suicide varies considerably with where you live in the world

4. High / low suicide rates don’t imply low / high happiness scores. In other words - suicide rates are poor predictors of happiness

Now go talk to your family and friends, have a walk in the nature or have a run - this is almost sure to make you more happy :)

Another attempt at a personal blog

29 May 2021

This is my n^th attempt at consistently keeping a personal blog up-to-date. Trying this out several years ago I failed, which taught me several lessons that will, hopefully, make things better this time:

Keeping it simple. My old personal website was too complicated to update. Say hi to procrastination.
Staying focused. What is my goal with a personal blog? I didn’t use to have a clear goal, which means I didn’t have an incentive to keep it up to date.
Keeping records of my learning helps with motivation. This is tied to the previous point. I want to have some sort of a virtual “diary” where I note down what I learned. “Use a personal notepad like Joplin”, I hear you say. The issue is - my notes tend to become messy, and by expecting some visitors, it makes me think twice (* 10) before publishing. Does it have any value to the reader?, Is it easily comprehensible? are just a couple of questions that I hope will improve my writing skills. As an aspiring Data Scientist, being a good writer is arguably a crucial skill.

And… that’s about it! I have kept it short this time to make sure that I can get started with something, however small (thanks Atomic Habits).

Older Newer

Przemek Zientala Notes on things