All posts on the blog RSS-feed

Performance in debug mode (Java/IntelliJ)

I'd like to highlight two steps to avoid performance issues when using breakpoints. Method/field breakpoints Is your application noticeably slower in debug mode, even when not in a break point? Did this problem start suddenly? Do computation-heavy tasks take especially long (like password hashing)? Then you may have some breakpoints that cost a lot of performance (why?). Breakpoints on lines of code are fast Breakpoints or fields and methods are slow (Breakpoints on declarations/comments don't work) In my experience, over... full post»

Example log4j2.xml (development)

For local development, I like to change some things in my log4j2 configuration. Make lines shorter, e.g. no worker names (just pid) or dates (just times). Prevent all logging to files, just console. Use colors for errors and warnings so they stand out. Filter out specific types of messages, and keep only specific other ones at debug level. Use buffering and async for better performance. So here it is! <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Configuration monitorInterval="30"> <Appenders> <Console name="Console" target="SYSTEM_OUT" bufferSize="100">... full post»

Clean build with IntelliJ and maven

Clean build - 70% of the time, it solves the problem every time. We'd like to understand every problem before solving, but sometimes it's just easier start with a clean slate and see if that works. So if you're using IntelliJ and maven, here is how to do that. Make sure you have no suspicious changes in your workspace. If you want to be really through, you can delete this directory, in order to make maven get all dependencies anew... full post»

Java performance: exceptions

This was a real performance problems recently, with some details redacted. It was found that filling in translated strings took up quite a lot of time. Translation requests are spread out all across the frontend, so no single step stood out, but with the help of YourKit, the problem was uncovered. As it turns out, the main problem was with this construct: for (int i = translationFiles.length - 1; i >= 0; i--) { try { ResourceBundle bundle = ResourceBundle.getBundle(translationFiles[i],... full post»

Java findFirst stream pitfall

Java streams have findAny and findFirst methods out of the box. And they works fine, they do exactly what they say: it gets one of the matches. But in my experience, often I don't want 'one of the matches', I want there to be a single match, and I want that one. The problem For example, finding the instances in a stream of customers var formExpressionInstance = getCustomers().stream() .filter(instance -> isDateInRange(ferenceDate, instance.getDateStart(), instance.getDateEnd())) .findFirst() If there are multiple matches, maybe... full post»

Java overload + vararg pitfall

If you are using overloads In Java, and one of them uses varargs, then there's a rare pitfall to be aware of. I ran into this a while back. There were two methods, similar to this: public void method( @Nullable Object first, @Nullable String second, @Nullable Integer third, @Nullable BigDecimal fourth, @Nullable Double fifth ) throws Exception { and public void method( @Nonnull Integer... varargs ) throws Exception { Then, someone added a sixth argument to the first method. Simultaneously... full post»

Some JetBrains IDE tips

Although they're expensive I really like the JetBrains IDEs. I use IntelliJ IDEA almost every day, and PyCharm and CLion quite frequently. Here are a few features I like! Multi-caret (info) to change multiple pieces of text simultaneously. You can place carets with a hotkey, but I especially like doing a search and putting a caret at each match. Much more flexible than search/replace. String manipulator: this plugin adds a lot of text manipulation on selected text under the hotkey... full post»

Show recently changed branches (git)

To show your recent branches using git: git for-each-ref --count=10 --sort=-committerdate refs/heads/ --format='%(HEAD) %(objectname:short) %(color:green)%(refname:short)%(color:reset) - %(contents:subject) (%(committerdate:relative))' Example output: * 14d5da9 parsing3 - Update version and readme (4 weeks ago) ae05513 dev - Version 0.4.0 (4 weeks ago) 6c695f0 parsing1 - Improve underscore handling in ints (4 weeks ago) 47fece2 parsing2 - Some progress, but pest output is not close enough to AST (5 weeks ago) 6c5aba2 webui-abandoned - No JIT for now (5 weeks ago) 7c60c75 site-url -... full post»

Symmetric encryption in Rust

I'm working on a file encryption utility (endec) in Rust. Encrypting and decrypting doesn't need much code, but it took me several hours of fiddling to arrive at the final version, so I thought I'd share the result. The crates I'm using (for the symmetric encryption part) are: aes for encryption secstr to store keys in memory more safely block-modes for block cipher utils So let's get right down to the code: use ::aes::Aes256; use ::block_modes::Cbc; use ::block_modes::BlockMode; use ::block_modes::block_padding::Iso7816;... full post»

Optimal counting system using fingers

It's sometimes suggested -perhaps jokingly- that we should use binary to count using fingers, as we'd get all the way from 0 to 1023! Much better than the 0 to 10 we get 'normally'. Of course, it's rather difficult for most people to know at a glance which number is represented by .|..| ||.|. (probably 256+32+16+8+2=314). This way of counting is most often suggested by programmers, because using this way is just using base 2, just like computers do. A... full post»

What is async/await?

I've been pretty active on Hashnode and inactive here lately. So I'm going to repost an answer I gave on Hashnode here, because I'm lazy like that... This is a conceptual explanation, not a beginner tutorial. Yes, async/await a way for the thread to not sit idle while waiting for something (usually IO like files, database or network). When you read a file synchronously, your code does not do anything while the disk is busy. This means that for programs... full post»

Some reasons I do not like javascript

It is no secret that Javascript isn't the most well-designed language, and these points have probably been made a billion times, but now it's my turn! Note that this isn't a criticism about a lack of expressiveness. You can create great things with javascript. But you can also footgun yourself very easily, which is what I'm criticizing there. Most languages are expressive, but good ones help you avoid mistakes instead of making them. You can pass more or fewer arguments... full post»

Visitor pattern and dynamic dispatch

The Design Patterns book by the infamous Gang of Four contains 23 object oriented design patterns. Some of them are kind of obvious (Adapter), or have become common knowledge since the book was published in 1994 (Iterator). My personal favourite is Visitor. Visitor "represents an operation to be performed on the elements of an object structure. Visitor lets you define a new operation without changing the classes of the elements on which it operations" [except you need to add an... full post»

Analyzing a single, trivial Java statement

Java isn't a bad language, but it is also not great, and it's age shows. Let's look at this simple Java statement: if (!Boolean.TRUE.equals(getIsRequired())) { It's very simple and common, and yet it includes a lot of the things I don't particularly like about Java: We need a getter for a trivial property access. I like the Python way with @property, or the Kotlin way with getters and setters being called implicitly. It is not explicit that getIsRequired is a... full post»

Prevent ID-mistakes using Rust types

Recently, I've been interested in designing software to make hard-to-find bugs reveal themselves at compile-time. An idea I've read about (I forgot where) that I found fascinating is the use of a special type for IDs in objects that represent database rows. Problem For example, there may be a User table and a Posts table in the database, with each post belonging to one user. Each of these would be mapped to a class/struct in the programming language. Perhaps each... full post»

Android apps I like (2017)

Sometimes I want to get something done on Android, and it takes an hour trying apps to find the perfect one. Other times, I just don't know I want something until I read about an app or someone tells me. To possibly save you some time, here are some apps I think are useful! Pocket This app is great. It lets me store articles for reading later. I often find something interesting to read, and then that article has a... full post»

Java primitives conversion & operations

First, booleans Booleans cannot be assigned to any other types, and you can't do any arithmetic on them. So true doesn't count as 1, nor does 1 count as true. No truthy and falsy in Java. Converting: a = b Some types can only be converted if you cast explicitly. These are usually the cases where a type with a wider or different range is converted to one with a narrower range. If you use explicit casting, you might be... full post»

Efficient overfitting of training data (Kaggle Bowl 2017)

During the Kaggle Data Science Bowl 2017, the leaderboard was based on only $198$ samples. The opportunity for overfitting was quickly understood, but initially only the naive option was mentioned, testing 1 submission per sample taking 66 days (still doable within the competition duration, but less than ideal). But then Oleg Trott got a perfect score in just 14 submissions! topic) I was really curious how he managed to do this. Together with Cas, I found out one way it... full post»

Linear model vs decision tree (in R)

Linear models vs decision trees I‚Äôve used the R statistical language a bit, a long time ago. It was my first real encounter with data science, but future encounters used Matlab and Python. But lately I‚Äôve been picking up R again, as it‚Äôs popular in the data science community. As practise / demo, I thought I‚Äôd do a simple exploration of the strengths and weaknesses of linear models versus decision trees. This was inspired by Claudia Perlich at kdnuggets. Let‚Äôs... full post»

"Invite" someone to ssh into your machine without port forwarding or firewall changes

If you want to let someone log into your machine or you want to log back into your machine from your server for some reason, normally you'd just set up port forwarding and connect to your IP. But there is another way: you can use ssh to create a connection to a remote machine, which that person can then use to log into your machine. Advantages: No need to set up port forwarding if you're on a network. No need... full post»

Rainbow colormap in javascript

Simple one one this time! How to make an RGB rainbow colormap in javascript? Example: This is useful, for example, when plotting charts with something like chart.js. This is the code for it. The first part is converting hsl colors to rgb ones, which you might skip depending on which browsers you want to support. The second part generates a rainbow by simply rotating the hue of a hsl color. The other parameters are optional: you can make the colormap... full post»

SCons Fortran example

I wrote about SCons several weeks ago. It took some getting used to, but I'm very happy with the result! It's found it to be more expressive and flexible than Make. Or at least non-standard things are much more straightforward; I guess Make can do almost anything with enough recursion and voodoo. I can't really comment on the speed; it's plenty fast, but there are less than 50 files. I took some time putting it together, partially because there aren't... full post»

Fortran closures

While trying to implement the Lancsoz method (to find the spectral range of a linear operation), I had to pass a procedure with some arguments bound to it (to keep things general). However, I read that Fortran has no closures, nor have I encountered something like Python's functools.partial (or javascript's .bind), nor **kwargs. So I got stuck for a little while, but some kind people at Stack Overflow came to the rescue as usual. It seems that something like a... full post»

Bye Make, hello SCons (Fortran)

Makefiles have been around for a long time and are used in many places. And not without reason: a straightforward dependency tree with one version of the target and a specific compiler and all the source in one directory is easy to setup and is handled well by (GNU) Make. It's a common situation, and if that describes your project, there's no need to switch. But the code I'm working with has a build script and Makefile that can make... full post»


I've started a draft about PyCharm 5 months ago. I really love PyCharm, it's the best! Everything is just so smooth. But it's hard to summarize what makes it great, and I don't want to list all the little details. I used Eclipse before, which wasn't bad, and there's not really any giant thing in PyCharm that Eclipse lacked (stability maybe). Rather, it's the hundred little details that add up to a much better experience. I also used Wing for... full post»

Dory backup removal

Daily backups, even if incremental, can fill up your disk space pretty quickly. So I've added some functionality to Dory to remove old backups. I wanted a way to keep a number of backups, with recent backups being closer together and past ones further apart. I've ended up using the reciprocal time-distance to each neighbour, multiplied by the square root of the age, as a 'redundancy score'. This way, if there are close neighbours (e.g. .two backups made on the... full post»

Minimal logging server & clients

Having solved backups, I attacked the the next important thing that gets in the way of building actual functionality: error logging. I've had some trouble setting up Sentry (possibly more my fault than Sentry's, but it does much more than I need so it's more work). I also tried Loggly which doesn't need server setup, but it captured mostly irrelevant syslog stuff and I'd need to pay a lot to keep logs for more than 7 days. So I figured... full post»

Reset static files cache each version (Django)

Ideally, you'd want your static files to be cached so that clients don't need to request them at all. In Apache, you can achieve this with: ExpiresActive on ExpiresDefault "access plus 1 year" Header append Cache-Control public But you also want the clients to request the new version as soon as anything changes. Since this can't be achieved with cache headers (they'll not request the files, so they never get the header), you can change the name instead. There's a... full post»

Dory backup scripts

I made a backup script that is working pretty nicely for me so far. It has several function, the last one being the essence of it. Make a dump of all databases (postgres & mysql) to zipped sql files. It deletes any that haven't changed since the last backup. Bring a list of git repositories up-to-date with with their remotes on the current branch, by cloning or pulling. (This is useful to have a backup of a non-bare repository). Make... full post»

Logging to files

Logging the output of a shell script is easy enough: ./ 1> log.out 2> log.err But if it the output is long and you find an error, you might want to know which output corresponds to it. Logging errors to one file, and output+errors to another is a little more tricky. And there are other potentially useful features: Truncate the log file if it gets too long. Keep the last part, and ideally whole lines. Add timestamps to each line... full post»

Ubuntu macros

The macro feature of Notepad++ was really useful. On Ubuntu I've had to do without Notepad++, and anyway it would be better to have something that works inn any program. I've found/made something today, and since StackOverflow locks all questions about such things, I'll post it here. First, install xmacro using sudo apt-get install -y xmacro. The basic commands are xmacrorec2 > "$macrofile" and xmacroplay "$DISPLAY" < "$macrofile". I've made a few small improvements: Assigned hotkeys so I can use... full post»

Development vs conflict

It's hard to have any appreciation of how big or small numbers are. Wolfram | Alpha helps sometimes, if you look things up. But it doesn't work for money numbers, so other times you conveniently find two numbers on the same site to compare: Global cost of conflict reaches $14.3tn, says report Foreign aid close to record peak after donors spend $135bn in 2014 To make it a little easier to compare, I'll write the zeros, and add another big... full post»

Effective altruism - intro

From an effectiveness point of view, I should probably have written this a week earlier, just before Christmas. I'm on time for the new year resolutions though! It's about effective altruism. Many people like to feel that they are contributing something to the world. Like making a donation to a charitable cause. But wouldn't it be great if you not only feel like you're helping people, but you actually are? How do you know if you hard-earned euros/dollars/... are accomplishing... full post»

Clementine music player

There is a lot of software to play music. I'm nowhere close to trying all of it, so take this recommendation with a grain of salt. I'm recommending Clementine anyway, since after finding it, I didn't feel the need to try other software after. The main reason I like Clementine is just that it does almost all the basic functionality very well. Things like Searching your music collection or a specific playlist Creating playlists from directories, searching results, etc Editing... full post»

Concentration noise

As a little addition to the concentration music post, I'd like to mention the situation where there are people talking around you and you can't help listening to what they say and it's very distracting. Instead of turning up your music really, really loud and ruining your ears, I find that you can tune out individual voices by simply adding more. This is well outside the realm of "music", but it can be useful. I put it on Youtube several... full post»

Concentration music: musopen

Does music help you focus? It depends on the person, the task and the music. Some people work well with the TV on, but I am not among them. I find that music where I listen to the lyrics distracts me, but other music can help me focus on boring tasks. Which should be understood as "helps my concentration to be marginally less terrible" for those things. For interesting tasks I don't need music. Music for which I don't listen... full post»

JSON: comments, order & numpy

Natively, JSON files do not support comments. Douglas Crockford removed them early on to prevent parsing directives, and the json standard is frozen now so they'll never return. Douglas Crockford is, of course, a pretty smart guy, so I'm not going to argue that decision. But like many other people, I want comments anyway. There's a package commentjson, but unfortunately it does not support Python 3. There are several other json tricks I like, so I made my own package... full post»

The blog!

#type-systems, #imadethis, #data-science, #travel, #backup, #example, #bash, #performance, #ide, #bug-prevention, #science, #storage


  1. Some reasons I do not like javascript @Mark A side effect of implicit type conversions https://kahde...
  2. Some reasons I do not like javascript @Mark Another interesting one
  3. Some reasons I do not like javascript @sander yea fuck javascript man!
  4. Android apps I like (2017) @Mark I've found and even better app for Chinese reading pract...
  5. Rainbow colormap in javascript @Mark The post has been expanded to include non-rainbow colorm...
  6. "Invite" someone to ssh into your machine without port forwarding or firewall changes @Mark Added details about how to use the established tunnel in...
  7. Rainbow colormap in javascript @Mark Now finally with color-coded source!
  8. Bye Make, hello SCons (Fortran) @Mark For SCons' own comparison against alternatives, have a l...
  9. Ubuntu macros @Mark Macro locks have been made more reliable, to prevent any...
  10. Dory backup scripts @Mark I've added some functionality to remove old backups! htt...