pondělí 19. března 2018

Dream castle in Heroes III by unit levels

1) Skeletons
While an average level 1 unit, the necromancy skill makes the skeletons a great choice for large maps with a lot of wandering creatures, which can be harvested.

Disadvantage: On tiny maps occupied only by castles, where the majority of the battles happen during the first two weeks, updated gremlins or sprites would be a better choice since they are better suited for castle conquests during the first weeks than skeletons.

2) Harpy
Updated harpies are masters of guerrilla attacks as the enemy does not retaliate and the harpies return to their original position unharmed. If combined with mass slow, melee units can have a tough time to take down the harpies.  If you dislike loosing units while cleaning the map, this is the creature for you. It was voted to be the second most favourite level 2 unit.

Disadvantage: While harpies are awesome units for map cleaning and castle defense against weak opponents because of their low attrition, harpies are lousy units for final battles where big unit losses have to be expected. If you expect early battles between the main heroes in the open field, wolfs with their impressive damage ability would be a better choice.

3) Elves
Each castle should have at least one shooting unit. Upgraded elves shoot twice during the turn, making their upgraded version highly desirable. Elves were voted to be the favourite level 3 unit.

4) Vampires
Once the count of upgraded vampires reaches ~20 units, they are awesome beasts for map cleaning as you may expect minimal losses due to their ability to resurrect from the flesh of their enemies. Rated the most favourite level 4 unit.

Disadvantage: Just like harpies, vampires are not the best units for the final battles. Furthermore, it takes two weeks to grow the necessary number of units from a single castle.

5) Pits
Upgraded pits can raise ridiculously expensive vampires in vast quantities from the slaughtered skeletons. Finally, you do not have to worry about skeleton losses during map cleaning as the lost skeletons get converted into vampires!

Disadvantage: Just like harpies, elves and vampires, this unit has to be upgraded to benefit from them fully.

6) Wyvern
To compensate the need to upgrade almost all lower level units, wyverns do not have to be upgraded to be of some use. Furthermore, wyverns' dwelling has very low prerequisite requirements (just a level 2 dwelling building), making it possible to recruit the wyverns during on the first day. Since they are also fliers, they can be of great use for early castle sieging.

Disadvantage: Champions, black knights and nagas are all beautiful creatures. But no other level 6 dwelling can be build on day 1. If we went with any other unit, we would not be able to recruit level 7 units on day 2 of the first week.

7) Firebird
While firebird is not the best level 7 unit, the fact that the dwelling is one of the cheapest (the second behind titan's), the unit itself is inexpensive, the weekly growth is above average, it is one of the fastest units in the game giving you the opportunity to apply mass haste/slow and decimate the opponent before he gets to the turn, and fire immunity allowing you to embrace Armageddon strategy (with the help of a support hero to store raised skeletons) make it a pretty decent choice. It is the only level 7 unit whose parameters were down-tuned in Horn of the Abyss. Furthermore, firebird dwelling building has only one prerequisite - level 6 dwelling building. Hence, if you play on normal difficulty, you just need to get 5 wood (or better, set the starting bonus to resources - wood and stone, and you do not have to worry about getting any additional resource or gold) and on the second day you will have 2 firebirds, 2 wyverns and 3900 gold in the pocket to spend it on whatever you want to (based on the situation you may want to keep it for capitol erection, or spend it on another hero, 8 harpies and 6 skeletons).

Since all the units would be from the same castle, there would not be morale penalty from mixing several units together. The castle would be usable for blitzkrieg strategy during the early phase of the game if played on normal difficulty level thanks to wyverns and firebirds. But the castle would also scale to long games due to the ability to harvest skeletons, which would be continuously converted into vampires. And because upgraded harpies and vampires simply do not die during the map cleaning, you may go into the final battles with vast counts of skeletons, harpies, and vampires. Furthermore, firebirds and Phoenixes are, contrary to many dragons, immune to Armageddon spell. And because Phoenixes are the fastest units in the game, you are almost guaranteed, that you get the opportunity to cast Armageddon before the opponent gets the opportunity to do anything. Hence, Phoenixes can be in some situations used for dragon slaying without any loss.

čtvrtek 14. prosince 2017

When do I write unite tests

  1. Whenever I am updating some piece of code that already works. By writing the unit tests, I make sure that the new code is at least as good as the old code. And that the new code does not brake things.
  2. Whenever I am fixing a bug. The unit test demonstrates that the bug was fixed. And it makes sure that the bug does not return in the future due to refactoring.
  3. Whenever I am not able to write a working code on the first attempt, it is a sign of complexity. And since it is said that debugging is tougher than writing a code, I want to make sure that some bug did not pass unnoticed.
  4. Whenever I am assigning a programming task to someone. A set of unit tests helps to communicate what do I want to get. And it nudges the assignee to use my interface, simplifying integration of the delivered code on my side.
  5. Whenever I get the code from the assignee. Reasoning: Whenever I am assigning a task, I generally provide just a very limited set of examples that the code has to pass because:
    1. Writing a comprehensive set of tests takes a lot of effort. Frequently more than writing the function itself.
    2. The assignee may find a much better solution to the problem that is incompatible with the original unit tests. When this happens, I genuinely want to use the better solution. But I do not want to waste a lot of my work.
    Unfortunately, when people write a code that passes all my tests they think that the code may not contain any more bugs. I enjoy proving them wrong.
  6. Before deploying the code. It happened to me in the past that my code was passing all the tests that I was using during the development. To enjoy the victory, I thrown at the code some newly generated data, expecting a beautiful result. But the code failed. Just like the assignee tend to overffit the "training" unit test set, so do I.
  7. Before publishing the code. A good unit test works as a nice illustration of how the code can be used. 

What to test in data processing code

Based on the past experience, data are dirty. If the current data are not dirty, the next one will be. Hence, it is reasonable to test the code for such scenarios. My checklist follows:
  1. Does the code do what it is supposed to do on a trivial example? The purpose of this unit test is two fold: it is a documentation of how the code can be used. And if this test fails on the client's computer but works correctly on your own computer, it is a sign of an integration problem.
  2.  If the code accepts numeric data, check following scenarios:
    1. Zero (a smoke test for division by zero error)
    2. One
    3. Negative number (not all algorithms accept negative values)
    4. Decimal number (not all algorithms accept decimals)
    5. NaN (not a number - permissible for example in doubles) 
    6. null (missing value)
    7. Empty set (no data at the input at all)
    8. Plus/minus infinity
    9. Constant vector (for example when variance is calculated and used in a denominator, we get division by zero)
    10. Vector of extremely large values and extremely small values (test of numerical stability)
  3. If the code accepts character data, check following scenarios:
    1. Plain ASCII
    2. Accented characters
    3. null (missing value)
    4. Empty set (no data at the input at all)
    5. Ridiculously long text
    6. Empty value ("")
    7. White space character (e.g. " ")

pondělí 4. září 2017

Keyboard shortcuts on MacOS

MacOs does some things right. And some not so much.

  1. A dedicated (and simple!) shortcut for displaying the configuration of an application. On windows, you generally have to fish for "setting" or "preferences" in the application menu. Not so on MacOS - it is always command+",".
  2. A variety of shortcuts for different screenshots.
  1. No dedicated keyboard shortcut for find and replace. On Windows, it is always ctrl+"r". On MacOS, it is application specific.  Sometimes it is command+alt+"f", sometimes command+shift+"f" and sometimes there is simply no keyboard shortcut at all and you have to use mouse to tell the app to perform the find and replace and not just find! If you wonder, both, F5 and command+"r" perform refresh.
  2. No keyboard shortcut for displaying context menu - you have to use right click on the mouse.

středa 30. srpna 2017

Not invented here syndrom

Whenever I find that my code can be replaced with a system call, I go a long way to replace my call with the system call because I generally trust more the sytem libraries than my own code. When it comes to third party libraries, I am conservative. Sometimes they are outright buggy, sometimes they are correct but get deprecated and removed. And sometimes the whole system gets so complex due to the gross amount of third party code that no one wants to deal with the mess anymore and the project dies off. Hence, I like to try new 3rd party libraries and study them. But I commit into using them extensively only after thorough consideration of alternatives and testing.

pátek 11. srpna 2017

A proper handling of nulls

I am familiar with two approaches how to deal with nulls in programming languages: either completely avoid them or embrace them. Languages for logic programming generally avoid the nulls (a great apologetic is given by Design and Implementation of the LogicBlox System). But languages that permit them should provide following facilities:

1) meta information about why the value is missing
2) nullable & non-nullable variables

For example, SAS got it right 40 years ago. In SAS, a missing value is represented with a dot. That by itself is not great whenever you need to print out the code or the data, because you never know whether that dot really represents a missing variable or it is just an imperfection of the paper or the printer. But it permits to easily define the reason why the value is missing:
    .                 // Generic missing value
    .refusedToAnswer  // Missing value with a metadatum
    .didntKnow        // Missing value with a different metadatum
Hence, generic algorithms can threat all missing values the same way. But if you want to treat them differently, for example because refusedToAnswer can have a vastly different meaning in a questionary than didntKnow, you can do it.

Furthermore, SAS provides optional non-null constraints on attributes, just like SQL. The only ward on SAS's implementation is that it raises exceptions only during the runtime, not during the compilation time as, for example, Kotlin does. Note also that nullability must be configurable for all variables. For example in C#, nullability is configurable only for value types, not class types. And this omission is a source of many null-pointer exceptions.

There is just one thing where I am not sure which approach is better. If we have a function sum, it can:
  1. Accept nullable variables and use some default strategy to deal with nulls (as SQL does).
  2. Accept nullable variables and blow during the runtime when null is encountered, unless some strategy is defined (R takes this approach).
  3. Have a dedicated sumnan function, which accepts nullable variables and takes a parameter, which determines, how nulls should be treated. Non-nulable variables get accepted by sum function (something like this is used in MATLAB, minus the type control).
The first approach is convenient to use. But potentially dangerous, because the user may never realize that a null leaked into the data and the conclusions are wrong.

The second approach is safer, because the user at least learns that something is wrong immediately when null is passed to the function without passing a strategy how to deal with nulls. The disadvantage is that it is a runtime check, not a static check. Nevertheless, programmers that are concerned about safety may use a lint to identify calls to functions with nullable variables without defining the strategy what to do with nulls.

The third approach is easier to validate by the compiler than the second approach. But the naming convention can be difficult to enforce.

Do you have some thought about this issue?

pondělí 31. července 2017

International Software Testing Contest - experience

Why I write about the contest: Since I am one of the winners of the first ISTC competition held in 2017, it is in my best interest to promote this competition in order to make famous.

The assignment description: The contest consisted of writing tests for two Java projects: Naive Bayes classifier and Prim's algorithm for calculation of minimum spanning trees. A copy of the assignment: download.

Strategy: In 2017, we were evaluated only based on the branch coverage (line coverage was ignored contrary to what was written in the invitation) and the mutation score. Only in the case of a tie, the count of test cases would have been taken into the consideration. Since no tie happened in 2017, the recommended strategy for the next years (under the assumption that the rules do not change) is simple: maximize branch coverage even at the expense of the count of test cases.

Furthermore, to maximize the mutation score you have to use asserts in your tests. My strategy was: print the result of the method into a console:
And use the printed output in the assert:
    assertEquals("textOutput".trim(), someMethod().trim());
I used trim methods because I did not want to deal with the errors caused by wrongly copy-pasting too many/too few whitespace characters. Is it a test strategy I would use outside of the contest? No way, because the toString() format can change anytime. It may not catch all deviations. And not all objects have to implement toString(). But at ISTC 2017 it worked reasonably well.

Idea: Use a generator of unit tests like EvoSuite. However, it is critical to make sure that the generated unit tests work even in the evaluation framework because a single non-compiling unit test will result in zero mutation score. If EvoSuite does not work with the evaluation framework, consider using Randoop, which is possibly less sophisticated than EvoSuite, but generates clean unit tests without any non-standard dependency.

Mutation testing: To measure mutation score you may use PITest. If you use IDEA, a nice plugin providing integration of PITest into IDEA is Zester. Following mutators were used to calculate the mutation score:
  1. Return Values Mutator
  2. Negate Conditionals Mutator
  3. Void Method Call Mutator
  4. Conditionals Boundary Mutator
  5. Increments Mutator
Warning: The branch coverage and the mutation score that you obtain from your favourite tools may not 100% agree with the scores reported by MoocTest - the tool used to evaluate these two metrics in the competition. Hence, if you can, train yourself against the framework used at the contest.

Another trickery I run into is that JDK7 was required. But because I also had JDK6 and JDK8 installed on my computer, I run into unpleasant clashes during the competition that has cost me 7 minutes of debugging. If you can, have just a single JDK installed on your computer.

Finally, the source codes were in their default packages and that was interfering with PITest (I realized only after the competition that Zester does NOT work if either the source code or the test code is in the default package). Hence, if you design an intricate plan how to win the competition, prepare also a simple fallback plan.