Assert Fail

Hetzner mentioned in a few places

webmaster@gewalli.se (Oskar Gewalli) — Mon, 23 Feb 2026 18:18:11 +0000

I’ve seen news about Lidl’s parent company launching a cloud unit a few times. Some YouTubers such as Milan Jovanović and Tom Delalande talk about using AWS and Azure alternative Hetzner. Here in Thessaloniki I’ve met people from IONOS (also a German company). I’m somewhat reminded about how Heroku had a great story about simple deployments. You then had to pay the cost if you got real use, so you would hear about how companies started on Heroku only to shift away because of cost.

DHH is probably the most famous technologist writing about moving away from using cloud providers.

There is a cost of moving from one provider to another, especially when you have a more complicated setup that uses many of the platform services. You might have to rewrite your software to match new infrastructure, which is why the competitive advantages are missing for cheaper alternatives due to lock-in.

Prompt injection is the new SQL injection only easier

webmaster@gewalli.se (Oskar Gewalli) — Sun, 22 Feb 2026 17:36:43 +0000

As I try out more and more of LLM assisted coding. The security issues, I feel like, they are taken extremely lightly. The security issue I am referencing right now is that like when you use untrusted input an adversary can take advantage. This is not the only security issue. Sam Altman deflects from the issue by talking about people “Google” for help on building biological weapons using their tools (i.e. that the training corpus contains sufficient information to allow for that). Other people have also expressed that it is a deflection from real issues.

Using LLMs trains you in becoming desensitized towards getting huge chunks of text pushed in your face. Having software that you cannot vet, because the amount of code has exploded, means that potential issues can be hiding in the snow. In the long term we get to a normalization of deviance make fertile ground for attackers. Or expressed differently there is a lethal trifecta for AI agents: private data, untrusted content, and external communication.

Using LLM assisted coding has many challenges. The way people use the tools changes as people discover ways that these tools can be made to work for them.

The interesting parts around security happens in how we use the tools. For instance there is docs around sandboxing for Claude, sandbox mode for Codex and security for GitHub Copilot.

For power users there is less focus on security as seen on for instance Kilo Code and Roo Code.

When I was but a young boy around 10 years of age (ca 1990), we swapped diskettes. I remember having to reformat my computer due to getting viruses. When I was in my 20ies (2001) I stayed at a landlady’s place in Lund, since I went to the university there. She had a computer that was noticeably full of malware. Later on when I moved to a student room, the local network had so many people with infected computers that if you plugged in a Windows computer that had just been installed, it would get infected before getting enough security patches from the net. I’m guessing that we are in a rhyming situation.

The world and US cloud providers

webmaster@gewalli.se (Oskar Gewalli) — Mon, 26 Jan 2026 17:13:39 +0000

For many years we have lived with a move towards the big cloud dragons. In my musings during more than 10 years ago I thought that we would see a move towards national datacenters due to security implications. There did not seem to be a lot pushback from the CLOUD Act in other countries.

The calculations around using US services have changed. AWS is probably feeling some of the heat from the disruption in the US government.

A coworker noticed that the integration goes deeper today than it did 10 years ago. You need an online account in order to use your Windows PC or your Mac. Or it might not have been removed right now, it is just difficult.

Service providers (such as Microsoft, Atlassian) are trying to move away from offering self hosted services. This could simply be because it is a pain (cost) to maintain old branches for old versions of software compared to offering only one version of the software.

It could be that as long as there is a high cost (hiring and reeducation) of moving away from the services offered compared to the alternatives makes transitions unlikely. Significantly higher price or further cancellations of legitimate business could change that equation.

Evaluating different programming languages for use with LLMs

webmaster@gewalli.se (Oskar Gewalli) — Sun, 11 Jan 2026 18:52:08 +0000

If we try to find some idea what language is better or worse for use with an LLM, we need to have some way of evaluating the different languages. I’ve done some small tests using different programming languages and gotten a rough estimate of how well LLMs work.

Experiences 2025

What I’ve seen is that for a bit bigger program:

Haskell takes time and converges slowly but requires less of my effort to fix.
F# converges relatively quickly but requires a moderate amount of effort to fix. I.e. requires more effort to fix than Haskell.
Rust can converge relatively quickly, requires moderate amount of effort to fix if you go functional, larger amount of effort to fix if you try to be more OOP.
C# can converge relatively quickly but requires keywords to be more predictable (for instance “clean architecture” for a web api), requires a bit more effort to to fix than F#. I’ve felt that depending on what keywords you use, you get different results.
Erlang can converge relatively quickly, though since I’m kind of rusty on Erlang I don’t know if it works as intended.
Python can converge relatively quickly. I have found that it requires more effort to fix than Haskell. I don’t know better keywords to use in order to get better results.

Note that these should be taken as small samples, so not representative of a larger set of tasks.

Process that I usually follow:

Use a template in the target language for the git repository (this is to ensure that we avoid some security issues and get faster convergence)
Give the LLM a very detailed specifications of a feature and have it add tests and code.
Have the LLM run tests and iterate on test failures (with some manual intervention)
Check that api starts, curl the api.
Read through the entire program and fix any obvious mistakes

Some of the time I have seen that the LLM goes into a long doom loop where it does not converge towards any solution.

Convergence

My own experience is that using functional architecture help LLMs converge towards correct solution more often. This might be because if you restrict side effects and use immutable data, it is easier to get working code. This aligns with my own experiences writing OOP style code in large code bases have benefited from using functional patterns to avoid hard to diagnose bugs.

Keywords

You might have noticed that I use keywords to influence the LLMs to output code related to a subset the training data. For C# I’ve mostly used “clean architecture”. For F# I’ve gotten C# adjacent solutions unless I’ve specified keywords such as:

Functional Core, Imperative Shell
Pipeline/Data Flow Architecture
Event Sourcing and CQRS
Redux/Elm Architecture

I’ve not done any closer comparison on how they compare to each other. I’ve seen that for C#, keywords are a must unless you work in an existing codebase with established patterns.

It would be interesting to see if we can find better keywords for different languages.

SWE-bench Multilingual

For a more formal approach we have SWE-bench Multilingual. In this they evaluate different programming languages based on a small sample of open source repositories:

It’s possible that the difference in resolution rate is because the dataset happens to contain more difficult tasks in some languages.

Note also the limited sample size:

42 repositories representing 9 programming languages

The conclusion that Python is the best language, does not align with what I’ve seen (I’ve used a lot of commercial models). I think this means that we could get better benchmarks to evaluate how well LLMs perform on programming languages.

Token efficiency

One interesting question is what programming languages can fit more functionality with less tokens: Which programming languages are most token-efficient?

If you go to the source for his post: RosettaCodeData you will find that a lot of the problems are not common to see in working software. The token per task graph does not align with my experience for say C# and Go where Go usually requires a lot of extra ceremony compared to C#. I had trouble with keeping Go within the token limit. This lack of semantic density could be due to lack of mature business oriented frameworks compared to Java/C#.

I have seen that Haskell seem to have nice characteristics for LLMs (converges more often to correct solutions with less need for me to intervene). Perhaps not only the point about token efficiency, but also that Haskell has better type system compared to say Java and C# and can give LLMs a more clearly defined area that it needs to output, a “narrower surface”.

We await more benchmarks

I would like to hear other peoples experiences. My feeling is that depending on how you think about programming, influences the process you use to get working solutions from LLMs.

Devaluation of work

webmaster@gewalli.se (Oskar Gewalli) — Tue, 06 Jan 2026 17:55:39 +0000

I have talked to multiple developers that have told me about getting LLM-generated pull requests. The pain is that we have a situation where you ignore the act of getting code to work and instead focusing on the optics of good enough looking code.

Instead of talking about how we ensure that the people who are tasked with doing the work, actually do the work, we talk about adding more LLMs as a solution to the problem of unproven LLM generated code.

The point of source code

Another point is that programming is not about having code for the sake of code: Programming language text is to have readable specifications of working software. Using languages like Python, TypeScript, JavaScript, Java or C# does not have any inherent value. The value of using common languages is that you have written text that can be modified and understood. Unreadable code is not a good thing. Machine generated code that no one wants to read, means you have instant legacy software.

Productivity

The amount of lines of code has never been a good measure for productivity. Remember “you get what you measure”.

I’ve seen indications that LLM coding tools make developers slower but they think they’re faster. Besides the indication that even when measurements suggest faster task completion, we get maintenance mess.

Disruption

Most software developers that I have worked with have learned by doing (writing code). Having a shift towards mostly reading code, could mean a greater comprehension loss than we expect.

Rather than an acceleration of software development we see a disruption, that gets hinted in Software Acceleration and Desynchronization. The ways we develop software changes (as it has done before). For instance code reviews in enterprise development was not a thing when I started working. It became more mainstream ca 5 years later (GitHub helped popularize review flow around 2010). We can expect the culture of software development to change in response to this new pressure.

Conclusion

If we devalue aspects of software delivery pipeline designed to give you working software without a solution to compensate for the loss of quality, we (software professionals) will get the blame for the resulting issues.

Slop bulkheads

webmaster@gewalli.se (Oskar Gewalli) — Thu, 01 Jan 2026 21:47:34 +0000

How can we ensure that we use the LLM to generate useful code/features and fix bugs?

I don’t think we can put the same expectations on LLMs as on junior coworkers. You hire junior developers in order to have people that are expected to grow and learn. LLMs come pre-trained (i.e. there is no point in teaching them as they are fixed neural nets after training). But anyway let us take the idea of having a multitude of “agents” to the logical conclusion.

Say that we have a bunch of teams working on producing a complete system. One reason to use microservices is to split teams into areas where they can iterate without the communication overhead of being fully integrated in the same code. We can argue that the same idea can be applied to isolate agent-contexts so that we treat LLMs as a team or a multitude of teams.

We know that it is easier to generate new code than to have old code suitable for LLM. Why not embrace this pattern? Create many small services. Can we minimize some of the risks associated with LLM generated code by giving smaller scope and permissions to the code bases that the LLMs are generating?

When we generate new code with LLMs we do know that LLMs are vulnerable to slop squatting attacks. This is besides all of the other security related issues related to LLMs. My guess is that if you start with a small known template, then iterate on that, you avoid some of the risks. When creating CRUD-approximate applications the amount of libraries that you need are known, why you can start with a good enough template.

What kind of infrastructure allows us to have fine grained control over the access that networked attached service have? Does this mean that there is a need to go deeper into K8S and similar in order to become more AI friendly?

Living with a machine ghost

webmaster@gewalli.se (Oskar Gewalli) — Wed, 17 Sep 2025 06:00:59 +0000

LLMs or vibing is one technology in a long row of technologies and products that have promised to make programming more accessible. LLMs from what I’ve read, seem more to be the ghost of a portion of the internet canned into a GPU-ready package.

Usually the promise is that “Now, you wont need developers anymore, your regular skilled employees will do the programming”. From what I’ve seen, looking at GUI programming it usually means you get a reduced programming environment without the tools programmers expect to use. If the problem or task is simple enough, the tool works well enough. When the complexity of the task gets out of hand, you will have programmers fix workflow or write “code”. If you are lucky, this means that you either don’t need programmers for the task or that you have bought yourself time until the programmers need to write a solution. The “hype” or “marketing” around the tool does not align with reality. Just because the salesman is lying doesn’t mean what he is selling is useless.

The companies selling LLMs are either telling you or implying that you can use them for everything.

AI Will Replace You
AI-driven automation will continue to eliminate traditional jobs but also create new opportunities and industries we can’t even imagine yet. / Jensen Huang

When it becomes obvious that this is not in line with reality they tell you:

It will be better
Artificial intelligence will reach human levels by around 2029. Follow that out further to, say, 2045, we will have multiplied the intelligence, the human biological machine intelligence of our civilization a billion-fold. / Ray Kurzweil
The pace of progress in artificial intelligence is incredibly fast. / Jeff Bezos

Then comes the next hook:

AI Won’t Replace You. A Human Using AI Will. / Reece Akhtar
AI will not replace humans, but those who use AI will replace those who don’t. / Ginni Rometty

This means that unless you live under a rock, you will be fed FOMO and FUD.

You then seem to have an ecosystem of grifters and scammers that sell you snake oil or LLM services. This would not be a problem had the service been useless. LLMs give you texts that, since they have ingested of huge amounts of content from online sources, should be statistically similar to what a human would have produced. This means that there is a nonzero chance that you can get useful “novel” output that compiles into source code that does what you want it to do. We are in a situation with a mix of disinformation and information. A coworker recently summarized this as “do not trust, verify”.

How do we navigate this space? My feeling is that in order to use AI generated source code, it is either sufficiently small that you can inspect it, or sufficiently locked down that you can trust it. LLMs are good at producing absolutely insane amounts of texts. This does not mean that they are more productive than a programmer that writes one line per day: you get a perverse incentive to inflate the numbers when lines of code is used as a productivity measure. My feeling is that you need to make sure that the LLMs are constrained by static type system and requirement size. You want the compiler help you verify the source code.

The best is to use functional programming. You can have programmers reduce their degrees of freedom in source code in order to make the code as easy to audit as possible. Some of the lessons from functional programming can be applied to other languages, so you don’t need to despair. Reduce the potential side effects and mutations. Use types and validations. You would need to figure out ways to constrain the LLM for languages like JavaScript and Clojure.

Code smells: when using Roo Code or Aider, the LLM generally tries to follow the patterns already in the source. Having complicated patterns or weirdness makes both you and the LLM confused.

DRY: Don’t repeat yourself takes on a new meaning with LLMs, since many of them like to repeat themselves.

Architecture and patterns: from what I’ve seen, LLMs perform better on common patterns and common looking source (style of code that looks like what is on GitHub). I’ve seen LLMs do well enough for common architectures. For more obscure patterns or in-house frameworks, the LLMs seem to have a much harder time to produce working code. This is probably the old new thing, as many prefer using common solutions over niche solutions.

Another way to reduce the potential output of a program is to write tests. The LLMs cannot debug or test the end product in the same way as a human developer or tester, so having tests becomes more valuable in order to verify that the code does what it is supposed.

These practices, tools and techniques are not new, but gain a new relevance if you want a working solution without unintended side effects.

Monorepo conversion continued

webmaster@gewalli.se (Oskar Gewalli) — Wed, 07 May 2025 16:06:37 +0000

This is a continuation of Sample monorepo conversion.

Since then we have learned some lessons. Also note that the monorepo conversions we have done have been of closed source code that we found was either tightly coupled or that we wanted to introduce tighter coupling between components without the overhead of shared libraries.

Git Subtree

Subtree the good parts

The good parts about using git subtree is that you can use regular git merge commands from old branches before the code became a monorepo. This is neat if you want to avoid a coordinated code freeze for when the conversion is done.

In the conversions we wanted to keep the history in order for change tracking to work.

Subtree the bad parts

On Azure DevOps the blame feature looks messed up while on GitHub and in Visual Studio blame shows what you would expect. Some of the links to merges also got messed up because we moved the repositories from the project into separate archive project. The links in Azure DevOps tasks got broken so they show up red. This is not so much a subtree issue as it is that we missed some of the details during the conversion. That Azure DevOps does not handle blame, is not really the fault of subtree.

You will have huge merge commits done by whoever got tasked with doing the conversion. These show up in history can look kind of ugly if you browse the commit history. Though to be honest, it is not that big of an issue.

Enter git-filter-repo

By using git-filter-repo you can rewrite the git history to make it look like your files have always been in a conveniently named subfolder of the monorepo.

cd sub-project-a
git filter-repo --to-subdirectory-filter ProjectA
cd ..
cd mono-repository
git remote add -f sub-project-a ../sub-project-a
git merge --allow-unrelated-histories sub-project-a/master
git remote remove sub-project-a

filter-repo good parts

We get a nicer looking history. The blame functionality in our current git host works.

filter-repo the bad parts

After rewriting history, you cannot do a regular git merge with old branches. You might need to recreate your changes by doing format patch in git.

Conclusion

Going towards a monorepo is not without hiccups. The pain points shared above are kind of minor. It did cause some confusion when one person tried looking for the commit when a feature got introduced and found what looked like truncated history in Azure DevOps. In a few years, perhaps this will be fixed by the Azure DevOps team.

Clean Architecture Comparison continued

webmaster@gewalli.se (Oskar Gewalli) — Mon, 18 Nov 2024 07:06:35 +0000

Background

A year ago I did a clean architecture comparison for some clean architecture implementation samples online. One of the conclusions was that I did not look into how much each implementation chooses to marry different frameworks and libraries.

Since then, we have seen some updates to existing solutions. We see one solution that has been archived eShopOnWeb, that was deeply influenced by @ardalis, that sample now lives on in nimblepros/eShopOnWeb. Some are not popular enough that it makes sense to keep them in this comparison. In this post we focus on C#.

Overview

Solution	Domain/Core	Application/UseCases	Comments
@amantinband	SmartEnum, error-or, MediatR, throw	FluentValidation
@ardalis	SmartEnum, SharedKernel, Specification, Result, GuardClauses, MediatR	…
@jasontaylordev	MediatR	EF, FluentValidation, AutoMapper	database framework dependency
@kgrzybek	MediatR	Dapper, FluentValidation	database framework dependency
@ivanpaulovich/clean-architecture-manga			Not updated

If we summarize this table we can note that some of the very popular solutions are dependent on database frameworks in the application layer. Having a database framework dependency is generally frowned upon in Clean Architecture.

Libraries in use

Many of the popular implementations take a dependency on popular libraries by @jbogard:

amantinband introduces:

@ardalis introduces:

These libraries are lightweight in that there is not much code or functionality in them.

We still note that many choose to include such libraries in the domain and or application-layer.

Conclusion

We see that at least one popular implementation called clean, choose to marry Entity Framework. Most of the implementations choose to include libraries. I would argue that marrying EF is something you need to be very much aware of in a Clean Architecture context. It can still be called clean, though with the addendum that it marries EF. We have a spectrum of clean depending on what is practical for a team to implement. I interpret this as, that there are missing patterns or language features for C# that makes mainstream implementation of Clean Architecture more of a chore.

Some of the libraries such as Result, SmartEnum and ErrorOr are needed because of missing language features in C#. These libraries are not needed in an F# solution.

Programmers in the age of Copilot or GPT

webmaster@gewalli.se (Oskar Gewalli) — Sun, 21 Jul 2024 19:17:16 +0000

Introduction

We have seen a rise in the use of non deterministic tools based on machine learning in recent years. Tools such as GitHub Copilot and ChatGPT have brought a change to how some programmers write code.

We have the following article: New GitHub Copilot Research Finds ‘Downward Pressure on Code Quality’ . You can find other papers around the impact of relying too much on such tools by searching for research papers.

My guess is that even though we have changes around tooling, that the impact will not be as large as the hype suggests.

Terms

I use the term programmer here broadly to mean anyone that writes code, implying also: software developer, software engineer, software architect et.c.. These terms mean different things depending on context.

Tools that have had big impact on programming in recent history

Unit tests

Unit tests ensure code performs as expected, catching errors early and facilitating refactoring. Despite their importance, many overlook their usefulness.

You need to think about how you write your tests and the impact they have as can be heard from DHH in test induced design damage.

The internet

The introduction of the internet drastically changed the accessibility of information for the average person.

Before the internet, learning resources were limited to books or physical digital media.

Code completion

Code completion is considered the norm today. I remember people telling me that it rots your brain. If code completion really rots the brain or not has become moot at this point for many mainstream programmers as the context around how we develop has shifted into making them indispensable.

Stack Overflow

With the introduction of Stack Overflow, we noticed that some programmers would copy paste “solutions” into their own code without reflection. We see this in the joke around the stacksort algorithm.

We know that people copy paste from Stack Overflow into their own source code, as can be seen in this article. I’ve seen an example of using Stack Overflow answer that contained desktop style code into a .net framework web application. The code worked when the programmer tested it locally, but failed when more than one person accessed the application. This illustrates the importance of understanding the context and functionality of code snippets before using them.

What I’ve seen over the years is that Stack Overflow has been a great resource for people to find answers to problems. We tend to forget that SEO made it difficult to find relevant information on the net even before the flood of machine generated content. You could argue that Stack Overflow fixed that since it promoted contributors (since then the site has gone downhill).

GitHub

GitHub has been a big boost to working with software. The main benefit has been that it has been a place where you can read source code and create issues on projects. Untrusted contributors have been given the opportunity to send pull requests.

Another side of GitHub is that it is a source of code examples that you can use as a base when you start your projects. The same dangers of downloading and using code from the net applies to source from GitHub as well.

Copilot

The way I use Copilot is often as a base and then modify the result to make it work. Sometimes when I’m lucky (or the problem is simple enough), Copilot will serve up the right answer right away.

I’ve seen Copilot deliver good enough simple regex, simple unit tests, simple React class components converted into function components, copy pasted code lines turned into function calls. I’ve seen that Copilot does well enough in some cases and fails in other cases.

Regardless of the tool, you always need to verify the generated code. Writing tests or having a short REPL feedback loop can help to confirm that it does what you intended it to do.

Alternatives

As an alternative to Copilot you can use ChatGPT as well as Ollama. You might also want to look into Agentic AI. It is yet too early to see how they will be used in the context of software programming.

Where do we end up from here

You have the same issues with Copilot as when searching for code online and reusing it. You need to verify that the code actually does what you think it does.

We have seen changes to how people program before. Junior programmers have done silly mistakes because they have tried to run before they could walk. One of the benefits of applying machine learning to programming compared to other areas is that you can get a feedback loop going by compiling and running tests.

Clean Architecture Comparison

webmaster@gewalli.se (Oskar Gewalli) — Sat, 02 Dec 2023 18:07:01 +0000

High level image of the clean architecture from Uncle Bobs blog post:

Below is a comparison between clean architecture solutions in C# and Java. Most of the samples are in C#.

High level overview

There are some similarities and some key differences between the samples. No general conclusions can be taken from this comparison since:

Closed source clean architectures are not represented (should represent the majority of the implementations).
Arbitrary samples based somewhat on GitHub popularity and what I have kept my eyes on.

Samples

Note also that the solutions I’ve picked are somewhat arbitrarily chosen. Some have a lot of attention on GitHub while I’ve kept my eyes on others for a longer time.

Sample	Use of Libraries/Structure	Grouping/Vertical slice
@CanerPatir/aspnet-core-clean-arch	WebApi, Infrastructure, Application, Domain libraries	Per use case grouping of folders in Application and WebApi
@ivanpaulovich/clean-architecture-manga	WebApi, Infrastructure, Application, Domain libraries	Per use case grouping of folders in Application and WebApi
@jasontaylordev/CleanArchitecture	Web, Infrastructure, Application, Domain libraries
@mattia-battiston/clean-architecture-example	Groups entities and use cases in core project	configuration, entrypoints, dataproviders, core libraries
@ardalis/CleanArchitecture	Web, Infrastructure, UseCases, Core libraries	Per use case grouping in Application
@dotnet-architecture/eShopOnWeb	Web, PublicApi, Infrastructure, ApplicationCore libraries	Aggregate, “Service”
@kgrzybek/modular-monolith-with-ddd	Api, Infrastructure, Application, Domain libraries	Per subdomain projects

Comparison

There does not seem to be a clear consensus around the naming of layers. Some projects are too simple to get into grouping into slices or subdomains, some of the projects (most notably modular monolith with ddd) shows how a more fleshed out structure could look (in the case that your web api only implements one bounded domain).

Sample	Command handler interface/base class	Query handler interface/base class
@CanerPatir/aspnet-core-clean-arch	ICommandHandler	IQueryHandler
@ivanpaulovich/clean-architecture-manga	IOutputPort/Named	IOutputPort/Named
@jasontaylordev/CleanArchitecture	IRequestHandler	IRequestHandler
@mattia-battiston/clean-architecture-example	-	-
@ardalis/CleanArchitecture	ICommandHandler	IQueryHandler
@dotnet-architecture/eShopOnWeb	-	Specification
@kgrzybek/modular-monolith-with-ddd	ICommandHandler	IQueryHandler

The solutions presented use a couple of different patterns. Most of the solutions use classes per action while some (such as eShopOnWeb) group all of the actions into one class per entity (in this case called service-classes).

We see that @jasontylerdev, @kgrzybek, @ardalis in their samples marries the framework MediatR. In the case of @ardalis we note that he marries the framework Ardalis.SharedKernel in his sample. In the case of the dotnet architecture sample eShopOnWeb that solution marries Ardalis.Specification. We can only echo what Robert C. Martin writes in his book Clean Architecture on page 292 that you have a extraordinarily asymmetric marriage when you take on a direct dependency on a framework. @ardalis is aware of this fact and write in the about text on his SharedKernel project the following:

Some useful base classes, mainly used with the CleanArchitecture template. Also, a template to make your own SharedKernel nuget package.

Sample	Anemic domain entities
@CanerPatir/aspnet-core-clean-arch	No
@ivanpaulovich/clean-architecture-manga	No
@jasontaylordev/CleanArchitecture	Yes
@mattia-battiston/clean-architecture-example	Yes
@ardalis/CleanArchitecture	No
@dotnet-architecture/eShopOnWeb	No
@kgrzybek/modular-monolith-with-ddd	No

The use of anemic domain model is not uncommon. A dissenting view from what Martin Fowler teaches is that you should embrace manipulating data instead of coupling data and behavior as seen in the book Data Oriented Programming. Note also that even though the domain might be too simple in many cases, I’ve given the authors the benefit of doubt.

Conclusion

From what I can see, there seems to be some convergence and some differences in naming conventions and patterns. These differences implies that it helps to document the source of your architecture and name conventions.

We have not gone into the how different samples around how they marry frameworks but simply noted that some of them do couple the domain to a framework. From a clean architecture point of view, being overly dependant on a framework, introduce significant constraints and asymmetries.

We could improve our understanding of how clean architecture is implemented by reaching out to different organizations implementing clean architecture, but that goes beyond the scope of a simple blog post.

Patch net7 dev container for net8

webmaster@gewalli.se (Oskar Gewalli) — Tue, 14 Nov 2023 20:17:07 +0000

Since there isn’t an official dev container for net8 yet, you can use the net7 dev container and the setup scripts mention on Microsoft Learn.

Instead of the base image:

FROM mcr.microsoft.com/devcontainers/dotnet:0-7.0

You can add the following to ensure that net8 is installed:

RUN wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh
RUN chmod +x ./dotnet-install.sh
RUN sudo ./dotnet-install.sh --channel 8.0 --install-dir /usr/share/dotnet

What does SOLID mean

webmaster@gewalli.se (Oskar Gewalli) — Sat, 19 Aug 2023 08:32:45 +0000

Sources

Principles

Single responsibility principle

Let us start with The single responsibility principle:

Gather together the things that change for the same reasons. Separate things that change for different reasons.

some see it as:

do one thing and do it well

The problem with this principle is that the definition is open for interpretation. The principle assumes sufficient experience around what kind of changes in requirements we can expect (something that depends on the people involved). I’ve worked with people that have a good grasp on how large a class/module or function should be, but the problem is that people have different expectations around what kind responsibility/concern code should have. It is a good thing to keep code focused on one thing, so the principle should be more of a reminder to break out or move code that does not fit. As Marco Cecconi in ‘I don’t love the single responsibility principle’ mentions, to avoid making a mess you need balance coupling and cohesion (not something you would expect a developer that has only worked for a few years to know). It can still be good to have it in mind.

Open-Closed principle

A Module should be open for extension but closed for modification.

Mentioned in The Open-Closed Principle. This is another principle that you need to have experience in order to apply in a good way. The risk is that it becomes as Dan North says

Cruft Accretion Principle

I don’t agree with Dan North. If you have smaller code blocks as mentioned in single responsibility principle you can garbage collect old code.

Perhaps some of the issues around this principle is that it becomes overly ambitious scope if taken out of context. If we pull back on the ambition and instead focus on some of the things he talks about in the article we can summarize as following:

The reason for the principle is that you want to avoid cascade of changes due to a single change of one code unit. That is

Allowing you to change behaviors without modifying code

An example is given how it is brittle to write code that uses “runtime type identification”. Why is that brittle? The reason it is brittle is because if you have a class that is not sealed in C# or final in Java or C++, someone can easily inherit and break old assumptions.

My take on the case of OCP is that you either make code:

closed: i.e. make sure that you cannot inherit by making the class sealed/final having an algebraic datatype
open: open for inheritance but have property/method to the class/type so that the you force child classes to take a decision

My feeling is that this principle makes most sense when talking about

Clear single responsibility principle violations
Inheritance in your business code. Many developers follow Composition over inheritance why you do not need to worry about open and closed in relation to inheritance in such code bases.
When you are developing framework/infrastructure code. It makes more sense to freeze framework/infrastructure behavior than it does to freeze business behavior (since businesses and people change).

Liskov substitution principle

The Liskov substitution principle is not formulated mostly by Uncle Bob, why it has a slightly different feel to it.

An implementation of an abstraction should be replaceable with other implementations of the same abstraction

or formulated in a different way

Code unit that publishes conformance to contract should conform to the contract

or in this way

A program that uses an interface must not be confused by an implementation of that interface.

IReadOnlyCollection suffers from a violation of Liskov principle as can be seen on Stackoverflow: ReadOnlyCollection vs Liskov.

We note that IReadOnlyCollection is more about what Dan North says:

acts-like-a, can-be-used-as-a

That is, if we send in a list to a method that accepts a read-only collection then we assume that the method wont mutate the list and the method assumes that we won’t mutate that list (during the execution of the method). This means that we are back at a more fuzzy interpretation rather than a strict principle. My feeling is that even though it is benign to let lists act like readonly, you are opening yourself up to trouble in the long run since it is a detail that is easy to miss.

Interface segregation principle

From object mentor article ISP.

Clients should not be forced to depend upon interfaces that they do not use.

or also formulated as

Keep interfaces small so that users don’t end up depending on things they don’t need.

The intention of this principle is that you should not require your code to depends on “fat” interfaces (or collection of unrelated interfaces).

Having to implement more methods requires more work. Introduces unnecessary work.
It is harder to know what parts of a larger interface is relevant to mock/fake when you are doing unit testing. Having smaller interfaces simplifies testing.

The problem with this principle is that there isn’t a sufficiently clear definition of when to apply this principle. In many cases depending on C# ICollection<> is better instead of a specialized interface with only the methods that you use. You rarely implement your own collections, rather you usually depend on existing framework implemented collections. If you instead try to limit this principle to business code (instead of infrastructure) you can end up with infrastructure code that makes your code hard to test when there isn’t any provided testing tools for the infrastructure.

Dependency inversion principle

From object mentor article DIP:

modules that encapsulate high level policy should not depend upon modules that implement details. Rather, both kinds of modules should depend upon abstractions.

or also formulated as

Depend in the direction of abstraction. High level modules should not depend upon low level details.

Why? My feeling is that he describes a situation with a solution with a mix of business logic and low level implementation logic. This principle makes most sense when you want to use an architecture such as adapters/hexagonal/clean architecture. Understanding that type of architecture requires you to be familiar with its culture and purpose.

If we are talking about two different things such as high level modules or detail being business code and low level modules being databases, integrations and hardware implementation. If we code directly against a specific database/integration/x86 architecture then we know that the code is less portable. Having infrastructure interfaces that includes business details means that the infrastructure is tied to that business domain. That is generally fine for solutions that only caters to one business domain but not fine for general purpose code such as say Entity Framework Core or Hibernate.

One of the big question I usually have around clean architecture is if you want to isolate yourself from the external infrastructure (such as Hibernate or Entity Framework Core) or if the usage of such frameworks can be seen as in line with the goal of having a way to isolate yourself from too much details about the database. The previous iteration of Entity Framework, called only Entity Framework instead with the “Core” tacked on, had a higher coupling with the database.

We should not design for reuse in our business domain, but rather if we are building software that is intended for many business domains. This means that by misunderstanding this principle it can lead you into writing “reusable” code that should never be reused.

Conclusion

My feeling is that SOLID makes sense. It does however require you to have worked as a software developer/engineer enough in order to get the experience needed to understand them. Working on complicated code and gaining enough experience will net you a feeling of what is common sense aligned with SOLID. Working in other projects that do not have sufficient complexity and enough business logic may well give you another view. I’ve talked with coworkers that have seen the failure mode of teams trying to apply the principles in a mechanical way. SOLID experiences what what Eric Normand denotes The Christopher Alexander Effect.

Data Specifications in Java and .net

webmaster@gewalli.se (Oskar Gewalli) — Fri, 07 Apr 2023 14:17:13 +0000

Background

In dotnet-architecture/eShopOnWeb I noticed the interesting usage of Specification classes. When I did some digging, I found that these are implemented through ardalis/Specification For those who are more familiar with the Java world we have Data Specifications. If you read Implementing Domain Driven design in the bibliography you find a reference to Martin Fowlers paper.

Sample based on Wikipedia

So if we look at Wikipedia we can infer the following code:

public class OverDueSpecification : ISpecification<Invoice>
{
    public bool IsSatisfiedBy(Invoice entity)
    {
        return entity.PayDeadline < DateTime.Now;
    }
}
// ...
var overDue = new OverDueSpecification();
var noticeSent = new NoticeSentSpecification();
var inCollection = new InCollectionSpecification();

var sendToCollection = overDue.And(noticeSent).And(inCollection.Not());

Higher level abstractions

We see that Ardalis Specifications collects System.Linq.Expression (note the lambda):

public class OverDueSpecification : Specification<Invoice>
{
  public OverDueSpecification()
  {
    Query.Where(entity => entity.PayDeadline < DateTime.Now);
  }
}

This can be seen as an alternative API to the interfaces used by EF like IDbSet<> and IQueryable<>.

Modules

Since the point of the pattern is that you should be able to separate the query specification from the data you could argue that the pattern is satisfied by having a module of specifications where you create the specification instances through method calls:

public static class InvoiceSpecifications
{
  public static IQueryable<Invoice> OverDue(this IQueryable<Invoice> query)
  {
    return query.Where(entity => entity.PayDeadline < DateTime.Now);
  }
}

public class InvoiceSpecifications
{
  public static JPAQuery<Invoice> overDue(JPAQuery<Invoice> query)
  {
    QInvoice invoice= QInvoice.invoice;
    return query.from(invoice).where(invoice.payDeadline.lastname.lt(LocalDateTime.now());
  }
}

Conclusion

The choice of using a class for a single specification or a method depends on how visible you want these specifications to be. By having a class in a separate file with a single specification, it becomes possible to browse in file system viewer. Some might argue that this introduces ceremony, but that depends on the preferences of the team (as can be seen in the eShopOnWeb example).

Running minikube on M1 and K8S tutorial

webmaster@gewalli.se (Oskar Gewalli) — Wed, 05 Apr 2023 17:00:00 +0000

As previously mentioned in the blog I use a M1 mac.

In order to help introduce others to Kubernetes I have done some testing of the K8S tutorial.

In order to try it out I first tried using the qemu driver for minikube. First gotcha was that the image gcr.io/google-samples/kubernetes-bootcamp:v1 does not work on ARM natively.

Let the Yac Shaving begin

Running minikube in docker

In order to get around this limitation you can run minikube in docker. Either by using colima or Docker Desktop with experimental Rosetta emulation.

minikube start --driver docker

Running the first part part of the tutorial works fine. You can create a deployment, you can view pods and nodes. The weird part starts when you forget that minikube is now running in a docker container instance and run:

kubectl expose deployment/kubernetes-bootcamp --type="NodePort" --port 8080
#...
export NODE_PORT="$(kubectl get services/kubernetes-bootcamp -o go-template='{{(index .spec.ports 0).nodePort }}')"
echo "NODE_PORT=$NODE_PORT"

Then try to do:

curl http://"$(minikube ip):$NODE_PORT"

expose intro

Since we are not binding all of the ports for minikube docker instance, the exposed port is not bound.

Instead you can do:

minikube service kubernetes-bootcamp --url

to get the url to the service.

Using qemu and building your own docker images

Since I started out with a simple qemu install I wanted to go all in the next round. I installed socket_vmnet and used:

brew install socket_vmnet
brew tap homebrew/services
HOMEBREW=$(which brew) && sudo ${HOMEBREW} services start socket_vmnet
#...
minikube start --driver qemu --network socket_vmnet

I then built images that uses both arm and x86:

First the Dockerfile:

FROM node:slim
EXPOSE 8080
COPY server.js .
CMD node server.js

based on kubernetes bootcamp Dockerfile

server.js:

var http = require('http');
var requests=0;
var podname= process.env.HOSTNAME;
var startTime;
var host;
var handleRequest = function(request, response) {
  response.setHeader('Content-Type', 'text/plain');
  response.writeHead(200);
  response.write("Hello Kubernetes bootcamp! | Running on: ");
  response.write(host);
  response.end(" | v=1\n");
  console.log("Running On:" ,host, "| Total Requests:", ++requests,"| App Uptime:", (new Date() - startTime)/1000 , "seconds", "| Log Time:",new Date());
}
var www = http.createServer(handleRequest);
www.listen(8080,function () {
    startTime = new Date();;
    host = process.env.HOSTNAME;
    console.log ("Kubernetes Bootcamp App Started At:",startTime, "| Running On: " ,host, "\n" );
});

based on kubernetes bootcamp server.js

My first instinct was to simply build the image and be done with it. But that is not as fun as going deeper. To take it further I wanted to grow the bananas to be used in the smoothie!

First I tried:

version='v1'
name="wallymathieu/kubernetes-bootcamp:$version"

docker buildx build -t $name ./$version -f Dockerfile --platform=linux/arm64,linux/amd64 --push && echo "Image $name built"

I got Error: multiple platforms feature is currently not supported for docker driver.

When running The buildx ls environment looked like it could handle amd64:

> docker buildx ls
NAME/NODE       DRIVER/ENDPOINT STATUS  BUILDKIT PLATFORMS
default *       docker
  default       default         running 20.10.23 linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
desktop-linux   docker
  desktop-linux desktop-linux   running 20.10.23 linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

I then found that others had solved this issue by simply creating a new buildx multiarch with docker container driver:

docker buildx create --name multiarch --driver docker-container --use

I could then build the images and push them to docker hub, as can be seen here.

Now I could continue on the kubernetes bootcamp tutorial using the custom images.

So instead of using the default 7 year old image (gcr.io/google-samples/kubernetes-bootcamp:v1) I wanted my shining new multi platform image built using the same code as in jocatalin/kubernetes-bootcamp:

kubectl create deployment kubernetes-bootcamp --image=wallymathieu/kubernetes-bootcamp:v1

And instead of upgrade your app using the old image, I used the custom built v2 image:

kubectl set image deployments/kubernetes-bootcamp kubernetes-bootcamp=wallymathieu/kubernetes-bootcamp:v2

Side-note

One thing that struck me was that why did they use gcr.io/google-samples/kubernetes-bootcamp:v1 instead of jocatalin/kubernetes-bootcamp:v1, I did get my answer when I tried using that image and could not connect to port 8080 on that image since the code assumes port 8081 while the dockerfile says port 8080 perhaps to have a reason to patch the image?

Conclusion

Successfully shaved the Yak! I got the tutorial working and only had to do a few smallish things in order to finish it.

Sample monorepo conversion

webmaster@gewalli.se (Oskar Gewalli) — Sun, 19 Feb 2023 11:47:28 +0000

Project with many repositories

This is a continuation of Monorepo versus Polyrepo.

This is a story about a project I worked on. We had some pain points related to many different source repositories.

Shared business domain library that contained shared business types
Shared library with a base parent entity (with common behavior)
Many services that dealt with different subdomains that uses a few shared libraries
Very complicated requirements with lots of nuances
Different persons had different areas of expertise
Generally the services are loose coupled
Thought out structure for each service and uniform style

At first, we thought that the shared code would stabilize. This did not happen. This meant a large number of pull requests with only package version changes.

We wanted to change code in the shared libraries along new core requirements. We had confusion around package versions due to the sheer amount of package changes. During retrospectives the team expressed frustration with the amount of unnecessary work.

Time to merge

We decided to merge the repositories into a shared repository for the .net code. This meant that we changed the NuGet packages to project references.

We also needed to support work on two major branches. For this particular business, there is a need to get legal to sign off on any major release. When we develop a major version we also bugfix the previous major version.

Git magic

We looked into using git submodule. We decided that it was not the right choice for the team: everyone would need to learn how to use it. Git submodule requires the team to learn additional commands to do pull and checkout in the main repository. We noticed was that you can use git subtree in similar way as git submodule. Difference is that only a few persons need to learn the extra commands.

You can fetch changes from different repositories into a merged repository.

Builds builds and builds

Our CI triggered a build for every new branch for every service.

Conclusion

Making changes to shared libraries become easier. We could use existing tools to make the transition (git subtree).

Running SQL Server on Docker with an M1/ARM CPU

webmaster@gewalli.se (Oskar Gewalli) — Mon, 06 Feb 2023 17:46:11 +0000

In 2021 my aging MacBook Pro (16GB RAM, I7) had issues with a swelling battery. The touch pad sensitivity became an issue. In order to replace that machine I opted for a MacBook Air (8GB, M1). Compilation speed (at least of F# code) on the Air is faster than the old machine, but it is not as responsive when I have a lot of things running at the same time.

In order to try out and showcase different technologies and services, I usually run Docker. Most images that I used where fine, however the one image that caused trouble was SQL Server as can be seen in the issue Does not work on Mac M1 #734.

For some cases I could use Azure SQL Edge. The troubles started when at work I started at a project using spatial types. I had the pleasure of getting a new developer machine (MacBook Pro M1) so I ran into the issue not only at home, but also at work.

To get around the issue I found that you can run container runtime in a different architecture (AMD) using Colima.

I tried out different configurations. What finally worked for me was to run:

colima start --arch amd --cpu 4 --memory 4
docker context use colima

This was inspired by a sample start configuration mentioned in the issue v0.4.1: x86_64 getting stuck with qemu64 cpu #288, but slightly different. These settings worked on a M1 MacBook Pro where I fiddled with the CPU and memory until it started working.

Right now, many months later we see that Docker for Desktop has an experimental Rosetta emulation setting as seen in Recap of issue Does not work on Mac M1 #734.

Conclusion

I’m still running Colima at work, but I’ve tried the Rosetta emulation at home and it works well enough (after a reboot).

Architecture and functional programming

webmaster@gewalli.se (Oskar Gewalli) — Sun, 05 Feb 2023 10:59:11 +0000

This post is a continuation of Unclean architecture.

Clean architecture

From what I understand of Clean architecture it’s an iteration of Hexagonal architecture also known as Ports and Adapters (see the Wikipedia page).

Clean architecture use of OO and code dependency

OO is the ability, through the use of polymorphism, to gain absolute control over every source code dependency in the system. It allows the architect to create a plugin architecture, in which modules that contain high-level policies are independent of modules that contain low-level details. The low-level details are relegated to plugin modules that can be deployed and developed independently from the modules that contain high-level policies.

Martin, Robert C. . Clean Architecture (Robert C. Martin Series) (p. 47). Pearson Education. Kindle Edition.

As we have seen around functional programming there are many ways of dealing with dependencies.

Compare to pure and impure

Compare Impureim sandwich with the hexagonal/clean/ports and adapters architecture

The question then is if the domain is supposed to be free from external code so that you have only code that you yourself understand or if it’s supposed to only be code without external IO/side-effects?

A clean as in pure functional domain without IO will be:

Testable
Independent of DB, external agency, UI

In the same way as a clean architecture. There might be implicit dependency on DB, UI et.c. due to features or flows that implies such. We even have Mark Seemann argue that functional architecture is Ports and Adapters.

Existence of clean code

There are people that argue that there is no such thing as clean code.

The problem is that low level vs high level is not a clear enough concept.

It feels like what Robert Martin is trying to describe is the business domain code without other peoples code (except when you really trust it), without code for IO like http, file access et.c..

You could perhaps argue that when trying to come up with a clean architecture for Haskell then one way to define such a scheme could be to you isolate your business code from any IO (something quite feasible in Haskell). The benefit of using some of the functional programming concepts is that they at least have a formal definition that can be verified.

My guess is that the problem is that for many senior developers/programmers clean code is related to feelings of cleanliness, so will be apparent to programmers that have gotten a feeling for it (my guess being that you would teach that through literary workshops).

Conclusion

FP concepts have come to have a broad influence over many mainstream core frameworks (see the functional concepts in Java and C# base libraries). We can argue that these concepts give us tools and definitions that help remove the fluff around the definitions. The downside is that the language of functional programming is often unfamiliar and feels foreign to many developers (why it makes sense to use the concepts without explicitly mentioning them).

Keeping a library in dotnet binary compatible for several years

webmaster@gewalli.se (Oskar Gewalli) — Thu, 01 Dec 2022 19:04:52 +0000

Background

I’m one of the people contributing to F#+. I got involved in the library around 2017 since I was looking for computation expression builders to clean up my F# code (after a few turns that turned out to be F#+).

Around 2018 we released the v1 version. Since then we have released minor versions of the library with new features and bug fixes.

In order to not break any users (such as our own usage) we try as far as possible to be backwards compatible.

Running tests against an old version

How do you do that? We have created a project that compiles a copy of the tests against the v1.0 release. Then we include those tests in the test execution with the current version of F#+. This means that the DLLs will be pulled up to the current version even though the project compiled against an old version.

The fsproj for the project that references F#+ 1 looks like:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>netstandard2.0;net461</TargetFrameworks>
    <RootNamespace>FSharpPlus.One</RootNamespace>
  </PropertyGroup>

  <ItemGroup>
    <!-- ... -->
    <PackageReference Include="FSharpPlus" Version="1.0.0" />
    <!-- ... -->
  </ItemGroup>

</Project>

While the actual test project will reference the above like:

<?xml version="1.0" encoding="utf-8"?>
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <!-- ... -->
    <IsPackable>false</IsPackable>
  </PropertyGroup>
  <ItemGroup>
    <!-- ... -->
  </ItemGroup>
  <ItemGroup>
    <ProjectReference Include="..\..\src\FSharpPlus\FSharpPlus.fsproj">
      <Private>True</Private>
    </ProjectReference>
    <ProjectReference Include="..\FSharpPlus.1\FSharpPlus.1.fsproj" />
    <!-- ... -->
  </ItemGroup>
</Project>

Note the reference to both the project that references F#+ 1 and a reference to F#+ currently being built (project reference).

This will allow us to see binary incompatibilities between v1.0.0 and the current version.

In order to read up on why this works you can read NuGet package resolution.

Binary compatibility with Statically Resolved Type Parameters (SRTP)

Since there is a lot of Statically Resolved Type Parameters (SRTP), the compiled version of F#+ will actually be binary compatible due to usage of SRTP being erased after compilation.

Since a lot of the API’s are not used directly by the user, but gets referenced by the compiled code we can declare overloads that are not used (by SRTP) in later minor versions of F#+.

This means that we can have methods as seen here:

type MapIndexed =
    // ...
    // Begin compat members
    // ...
    static member MapIndexed (x: 'T []      , f            , [<Optional>]_impl: MapIndexed) = Array.mapi f x
    // ...
    // End compat members

but that the new versions have the following signature:

    // ...
    static member MapIndexed ((x: 'T []      , f            ), [<Optional>]_impl: MapIndexed) = Array.mapi f x

Note the extra parenthesis. The compatibility members are referenced by the v1 version, while an updated version with overload fixes uses the lower form. Since user code will reference select the correct overload through SRTP, the compiled code just need to have one of the above.

Why does it get inlined? It gets inlined due to the fact that we use to select the correct overload is marked with the keyword inline in F#. The method that is use to select the correct overload is the following:

    static member inline Invoke (mapping: 'K->'T->'U) (source: '``Indexable<'T>``) =
        let inline call_2 (a: ^a, b: ^b, f) = ((^a or ^b) : (static member MapIndexed : (_*_)*_ -> _) (b, f), a)
        let inline call (a: 'a, b: 'b, f) = call_2 (a, b, f)
        call (Unchecked.defaultof<MapIndexed>, source, mapping)

The above code picks either the type Indexable contract from the incoming type or from one of the overloads defined in the MapIndexed class. In F# you don’t have method overloading for modules. For classes we do have it. The reason why is that F# needs to be compatible with C# parameter overloading for classes.

This has allowed us to do what normally would mean an incompatible change of the API, but let us keep the compiled binary signature (minus SRTP usage through say Invoke).

Hiding but keeping old APIs

By adding the attribute EditorBrowsable(EditorBrowsableState.Never) we tell the editor to not show types and methods when using intellisense.

One of the examples are the v1 Builders module:

[<Obsolete("Compatibility with v1"); EditorBrowsable(EditorBrowsableState.Never)>]
module Builders =
  //..

Conclusion

Using these facts of F#:

Statically Resolved Type Parameters in F#
Inline methods in F#
Implementation of C# overloading in F#
EditorBrowsable
NuGet package resolution

we have been able to keep a major version going for way longer than I would have thought possible while still being able to accommodate significant changes.

This also goes to show that F#+ has been the library that keeps on showing interesting usages of .net that one might not think possible (in .net).

Way of working and method

webmaster@gewalli.se (Oskar Gewalli) — Mon, 26 Sep 2022 16:24:22 +0000

How do we develop software

How do we arrive at working software? Some of the ways I’ve seen are:

Use of known working patterns, techniques, language features, libraries: Restrict what everyone does to only the patterns that are known to work. The assumption is also that by using a small subset of tools it is going to be easier for juniors and generally 99% of all programmers to join.
Use logical reasoning based on known properties of written code and the implied effect of changes.
Manual testing of business logic by a user or a tester or developer trying to act as a user in order to verify that the solution work.
Manual testing with a debugger in order to verify that smaller parts works as intended.
Automatic tests: program executes small parts of the program in a somewhat controlled manner to verify that it works as intended (and that no regressions have occurred).
Divide and conquer: Split a system into smaller parts in order to make it easier to make the individual parts.

Not every software shop agrees on how to test software. GUI software is usually dealt with in a different manner compared to server software.

Restrictions

Developers can be helped in their reasoning by restricting what the code and developers are allowed to do.

Common social methods (that I know of) used to restrict a solution are:

Avoiding mutable state : As seen on Microsoft and Oracle
Avoiding hidden side effects (such as through global static variables).
Consensus around architecture of a solution
Freezing code: avoid any change of parts of the written code. You can employ the pattern for longer than is suggested on the article about Freeze on Wikipedia.
Need approval before applying changes. See Pull requests section on Distributed Version Control.

These methods are not applied in every codebase.

Mechanical restrictions

Common tools to restrict what the code is allowed to do and how it is written are:

Statically-typed languages such as C#, Java as opposed to for instance JavaScript. There are many such languages.
Static code analysis tools. It is common to use linters for JavaScript and TypeScript, but there are others.
Executable tests. For instance Unit Testing.
Architectural analysis tools. We have seen the use of specifications as tests in ArchUnit.

I do mean that not all of these restrictions are applied in every development shop.

Scientific method

How does this compare to the scientific method?

Automatic/manual testing is a way to verify behavior of some code given some known state.
A known working pattern in programming could perhaps be seen as a hypothesis that has been proven to work for some use cases.
The use of logical reasoning seems very much connected to the scientific method.

Existing code

In many cases it takes many years and you have multiple people and involved. The task of keeping software working as intended then also involves knowledge sharing.

New people (to the software development team) run the software with a debugger and trace logging in order to get an understanding for how the system works or is intended to work. Debugging takes a lot of time, so you would limit yourself to only what you think you need.
New people read the written code in order to get an understanding for how the system works. This is a very difficult thing since the amount of code written is usually large enough that people need to develop speed reading skills and be able to summarize huge chunks of text. Different styles and different lingo can make the task even more difficult.
Use of written documentation in order to document intended behavior. This is notoriously hard since writing documentation requires writing skills in another language than the programming language. Even if you write documentation in your native language, you might not be a good writer. I could imagine that a good poet or song writer does not automatically become a good novelist. The pressure on delivering can make you ignore some parts (such as the documentation).
Use of testing in many ways in order to verify that the system still works after new features have been added. This is sort of like when you add plumbing into an already built house. If you are not careful you might ruin the house while adding the feature.

What I’m trying to say is that there are many ways that people have been going on writing software. It is not difficult to arrive at working software that no one can read. I’ve seen it happen due to too much text or written in a language/pattern people are unfamiliar with. Common reasons why that might happen can be:

The person is a junior developer.
The code is old. It’s written in a language or in a style no longer in common use.
Written in a style from a movement that the person is unfamiliar with.
Written in a haphazard way.
Wall of text (wall of code). The amount of code/text makes it difficult to grasp.

Literary methods

This gets us into the question how do you write.

Known language. An example of this is Ubiquitous Language.
Known patterns that help people understand what the program is intended to do.
- map/filter/reduce or select/where/join
- factory/observable/command et.c. inspired in part by Gang of Four.
Structuring the code in order to be able to be able to find different parts.
Strategy around important and less important text.
Splitting up the code into smaller chunks that are meaningful on their own.

This is mostly my interpretation of written code. I usually try to understand the code and the context it was written in. Many words and patterns make sense if you know the background of the programmers involved.

Conclusion

The point I’m trying to give you is that there are many ways of writing working software. The common parts are related to social methods, formal methods and literary methods.

Unclean architecture

webmaster@gewalli.se (Oskar Gewalli) — Thu, 22 Sep 2022 05:08:43 +0000

This post is a continuation of Small and simple business applications using SQL.

Clean architecture

From what I understand of Clean architecture it’s an iteration of Hexagonal architecture also known as Ports and Adapters (see the Wikipedia page).

Why would you want architecture? The main purpose is to deliver a working solution to the customer. How do you arrive to that? Architecture is a tool that enables developers to reason about how the system works.

Sample code

A sample succinct implementation is the F# : Onion architecture in a nutshell. You could translate that into Typescript.

Unclean architecture

It’s nice to have a clean architecture. Though as Uncle Bob says, there is a cost. Some people have argued against these parts. You then arrive at another important architecture pattern that we can call the Unclean/untidy architecture (it’s not totally dirty but not clean as well, I’m unsure if to call it unwashed or unkempt):

Dependent on some Frameworks
Dependent on some Database
Dependent on some external agency
Somewhat Testable

The most important such architecture must be Rails since you write code that is very dependent on framework (Rails), dependent on that there is a SQL-database. As long as you keep it somewhat tidy you can test using sqlite locally and integration test your controllers. You can configure your app to spin up a local db and test against that (even the same type of DB that you use in prod).

DHH argues there are too many downsides of using such an architecture.

As long as you keep your app relatively small, this can be quite OK. For larger solutions, you start feeling the pain. Your milage may vary.

Another way of writing unclean applications is to use a thin API wrapper around SQL. This style of untidy architecture can work well in some cases.

An example of small dirty app is to write a small service that does a mix of external and internal calls in order to have an app that can be in something like a smallish program wrapped by a docker container. The idea is that having an explicit dependency on one specific DB and no safety net does not matter as long as it’s easy enough to recreate a development experience using containers.

Accepting grime or dust

When is it a good idea? Even Uncle Bob says that:

There are some frameworks that you simply must marry.

Martin, Robert C. . Clean Architecture (Robert C. Martin Series) (p. 295). Pearson Education. Kindle Edition.

Do you consider Rails, EF core as frameworks you wed? As long as the app is still testable with an in memory db using EF core, can it still be considered clean (as long as it follows the other parts)?

What does it mean to have a totally clean architecture? Is the concept of clean architecture easier to communicate than:

in order to test you need to have a very custom SQL database up and running (something like backup and restore of an existing DB)
in order to compile the code you need to have several libraries installed on your machine (globally installed) and installed in a very specific order
in order to run the code you need to have specific versions of a couple of programs installed in specific locations
any triggering of payments, causes real invoices to be sent to real business and inventory to be updated

I think such a solution is harder (in some sense) to work on.

The main benefit of using a dependency directly trickles down to the advantage we get from being able to swap out the dependency (in tests and business code). If a dependency has a complicated contract hard to capture using a simple fake. An example is when the behavior of ORM creates an implicit dependency on using the real thing to verify your business logic. It’s sort of like a cohabitation relationship. You are not married, but the practical difference might be minor.

Conclusion

Clean code/architecture probably suffer from what Eric Normand denotes The Christopher Alexander Effect: that the advice around clean architecture/code is almost obvious to some senior programmers while unhelpful for others.

In order to know when it’s a good thing to accept an Unclean architecture, you want to have some ideas about what a Clean architecture is. I’m thinking poetic license may apply for architecture as well. The main benefit of a somewhat unclean architecture is the reduced amount of code and ceremony (sometimes a very difficult choice to make).

Small and simple business applications using SQL

webmaster@gewalli.se (Oskar Gewalli) — Sun, 03 Apr 2022 08:42:31 +0000

Say that you want to write a minimal API as seen in Can .net compete with NodeJS. How would you structure such an application? You would end up with something like 0-1 layers in your application.

Some of my previous coworkers embrace these patterns in order to focus on business value. In this post I’ll try to get into how you could make such a system without going into detail (pointing instead to examples).

Embrace SQL

One way to write simple API’s is to embrace SQL when you feel that it’s appropriate. See for instance It’s Time To Get Over That Stored Procedure Aversion You Have.

If you have need a low ceremony application that does not need much in the way of server side logic, there is no need to over-engineer the application.

There is a sample in NodeJs that is also ported to TypeScript. Just make sure to keep things parameterized.

SQL and Document databases

When you read a big object graph and store the same structure in one go, your application logic can be significantly simplified if you store that structure in one go without having to normalize the structure into a SQL view of the world. You could also view PostgreSQL as a DocumentDB, so there might not need to be a hard divide between the SQL and the DocumentDB view of the world.

Note for instance the tutorial from MongoDB where the JSON is as follows:

{
  "first_name": "Paul",
  "surname":"Miller",
  "cell": "447557505611",
  "city":"Lenden",
  "location": [45.123,47.2321],
  "profession": ["banking","finance","trader"],
  "cars":[
    {
      "model":"Bentley",
      "year": 1973,
      "value": 100000
    },
    {
      "model":"Rolls Royce",
      "year": 1965,
      "value": 338080
    }]
}

A decomposition into SQL could be:

create table customers(
  customer_id int identity(1,1),
  first_name nvarchar(255),
  surname nvarchar(255),
  cell nvarchar(100),
  city nvarchar(50),
  location POINT,
  primary key (customer_id));

create table professions(
  profession_id int identity(1,1),
  description nvarchar(255),
  primary key (profession_id)
);
create table customer_professions(
  profession_id int not null foreign key references professions (profession_id),
  customer_id int not null foreign key references customers(customer_id),
  primary key (customer_id,profession_id),
);

insert into customers (first_name,surname,cell,city,
                       location)
               values ('Paul','Miller','447557505611','Lenden',
                       POINT(45.123,47.2321));
insert into professions (description) values ('banking'),('finance'),('trader')
insert into customer_professions (profession_id, customer_id)
select p.profession_id, c.customer_id
from professions p where p.description in ('banking','finance','trader')
from customer c where c.first_name = 'Paul' and surname = 'Miller' ... -- assuming only one such customer ...

The structured nature of SQL also implies that you would need a bit more code in order to deal with the data.

Both Postgresql and Microsoft SQL Server support the use JSON. You can write views on top of JSON information in order to select the information. For migrating the schema you could take a note from RavenDB.

Testing it

When you write tests for a system that does not shy away from using the database as its core:

Setting up a development DB in a docker container can reduce complexity when setting up a development environment.
SQL Migrations makes it easier to keep databases in sync (and will incidentally make it easier to deploy your code).
There is an overhead when using a real database for tests. Though that overhead can be small. For systems that span over 200+ tables that overhead should not be underestimated (though such a system should probably be decomposed for many reasons).

Schema migrations

Being more structured when it comes to migrations will (from what I’ve seen) pay off, since it means that you can offload work to computers such as automated deployment, testing and the act of keeping databases in sync.

Some of the migration frameworks I’ve used (note that all of them allow for raw SQL migrations in some way or another):

In the .net world:

EF migrations : These migrations are nice enough if you let the object-model drive your database model.
Fluent migrations : Nice tool that makes it easy to write up and down migrations when you use their API to write simple enough migrations.
DbUp : Nice tool when you want to focus on SQL (you write sql-files).

In the Ruby world:

Rails migrations : Nice migration framework that make it easy write simple reversible migrations when you use their API for simple enough migrations.

Abstractions and schema

Keeping fewer layers means that there are fewer abstractions that people need to learn. This can help make the development more accessible to newbies. Having 200+ tables does not make for a pleasant experience going into a system. I’ve spent a lot of time at work to make sense out of such schemas. In many SQL implementations you can have multiple schemas in the same database in order to make things easier to understand.

The dreaded Monolith

In a sense many of the systems that you start to dread start out as small systems. Adding small parts to a system without looking for ways to decompose the system into smaller systems is how you end up with such a mess. There is no need to over-engineer your system until the pain of maintaining the system gets impacted (though note the boiling frog metaphor when it comes to the gradual increase in technical debt).

Conclusion

There are some security implications of embracing SQL, but those can be managed. As long as you keep the system smallish you can have nice enough time for developer written tests. There are as always a challenge around knowing when something has grown too big.

Validation and mediator

webmaster@gewalli.se (Oskar Gewalli) — Sun, 13 Mar 2022 11:59:41 +0000

This example is loosely based on how the API looks for aggregates in Axon Framework.

We also base some of the code around examples of ValidationBehavior found online.

The reason why I’ve implemented these examples is in order to showcase how you can cut down on some of the ceremony around using a generic mediator interface implementation together with fluent validation logic (pipeline behavior in MediatR parlance).

Note that I’m not making a value judgement around if you should use MediatR or not, if you should use AOP or not.

Assumptions

We assume that we want to use some validation API (here we use FluentValidation).

We assume that the domain have most of the business code in the entities.

We assume that we want to use MediatR (you could substitute MediatR with a message queue abstraction).

We assume that we use some sort of unit of work (if we use EF Core that would be DbContext, while it would be session if we use NHibernate).

STEP 1

First step is to create the classes that will implement the MediatR IRequestHandler:

FuncCreateCommandHandler
FuncMutateCommandHandler

You could implement these classes in the following way:

    class FuncMutateCommandHandler<TEntity, TCommand, TResponse> : IRequestHandler<TCommand, TResponse>
        where TCommand : ICommand<TResponse> where TEntity : IEntity
    {
        private readonly Func<TEntity, TCommand, IServiceProvider, TResponse> _func;
        private readonly IServiceProvider _serviceProvider;

        public FuncMutateCommandHandler(Func<TEntity, TCommand, IServiceProvider, TResponse> func, IServiceProvider serviceProvider)
        {
            _func = func;
            _serviceProvider = serviceProvider;
        }

        public async Task<TResponse> Handle(TCommand cmd, CancellationToken cancellationToken)
        {
            var repository = _serviceProvider.GetRequiredService<IRepository<TEntity>>();
            var keyValueFactory = _serviceProvider.GetRequiredService<IKeyValueFactory<TCommand>>();
            var entity = await repository.FindAsync(keyValueFactory.Key(cmd));

            var r = _func(entity, cmd, _serviceProvider);

            return r;
        }
    }

See FuncMutateCommandHandler

Note that the signature of the delegate is: Func<TEntity, TCommand, IServiceProvider, TResponse>

The base responsibility for these classes will be to retrieve and store the business entities as well as provide an opportunity to inject validation logic (since we implement a generic interface).

Since they take a func it’s trivial to create a fluent API in order to register how the entities :

Given_fluent_registration_of_handlers

The setup then becomes the description of how we tie together the business entities (Person).

        services.RegisterHandlersFor<Person>()
            .UpdateCommandOnEntity<EditPersonCommand,Person>((entity, cmd, svc) => entity.Handle(cmd, svc))
            .CreateCommandOnEntity<CreatePersonCommand>(Person.Create)

STEP 2

Since we have the initial classes needed to write automatically created registrations based on attributes we can now create our first such implementation.

We note that what we need to achieve is to write code that registers the above handlers. Since we want to be efficient we write a Linq Expression that essentially does:

    new FuncMutateCommandHandler<TEntity,TCommand,TResult>((entity, cmd, svcProvider) =>
        entity.`MethodInfo`(cmd, svcProvider), outerSvdProvider)

This code is not too difficult to write. See the implementation.

Resoning 1-2

We have a solution that deals with IServiceProvider and a lambda. Since given a service provider we can always get a service, this essentially means that we have the needed building blocks to adjust the solution into taking multiple services.

Step 3

Since we have the building blocks we need to find places where we have registrations that matches the pattern:

    [CommandHandler]
    public ... Handle(EditBookingCommand cmd, IAService services, ...) ...

In order to find such we can write some simple reflection based code (in the example we scan for instance methods that does not take service provider as a parameter).

Then the heavy lifting is to write an expression that does:

    new FuncMutateCommandHandler<TEntity,TCommand, TResult>((entity, cmd, svcProvider) => entity.`MethodInfo`(cmd,
        svcProvider.GetRequiredService<IAService>()
        ), outersvcProvider)

See the implementation.

Conclusion

I’ve talked to some coworkers that have expressed a feeling that there is too much magic going on with a decomposed solution that uses generic interfaces. This is similar to the criticism around AOP.

We have around 400 lines of infrastructure glue code to tie the code together. The main difficulty in the implementation is around writing Linq Expressions. The reflection code can be seen as similar to how you would write queries against a database, so might not be so far fetched. The Linq Expression code is somewhat abstract, that might be more difficult to understand.

If you want to write C# such that you find MediatR useful, then the complexity could be hidden as library or framework code. You cut down on a lot of copy paste and ceremony for implementing command handlers.

Can .net compete with nodejs

webmaster@gewalli.se (Oskar Gewalli) — Wed, 10 Nov 2021 06:42:31 +0000

NodeJS applications can be a single file with minimal overhead:

const express = require('express')
const app = express()

app.get('/', (req, res) => { res.send('Hello World!') })

app.listen(3000, () => { })

How does this compare to what we have seen in the .net world?

For .net 5 C#

HelloWorld.csproj (1)

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

</Project>

Program.cs (1)

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

namespace HelloWorld
{
    public class Program
    {
        public static void Main(string[] args)
        {
            CreateHostBuilder(args).Build().Run();
        }

        public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .ConfigureWebHostDefaults(webBuilder =>
                {
                    webBuilder.UseStartup<Startup>();
                });
    }
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        public void ConfigureServices(IServiceCollection services)
        {
        }

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseExceptionHandler();
            }
            app.Map("", b => b.Run(async context => await context.Response.WriteAsync("Hello world")));
        }
    }
}

the amount of ceremony has been reduced significantly compared to asp.net 4.

.net 5 F#

In the F# space we have seen such libraries as Suave, where you can write an fsx file containing:

#r "nuget: Suave, 2.6.1"
open Suave

startWebServer defaultConfig (Successful.OK "Hello World!")

Note the relatively recent NuGet integration into the F# language that makes for a great scripting experience.

In Giraffe there has been a larger fluent configuration due to the fact that it is implemented using asp.net core as the host.

Towards minimal APIs

For C# the overhead of writing a minimal HTTP API would even more ceremony compared to F#. Perhaps due to the fact that there is competition in a wider world we see initiatives such as Feather HTTP where you have:

using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;

var app = WebApplication.Create(args);

app.MapGet("/", async http =>
{
    await http.Response.WriteAsync("Hello World");
});

await app.RunAsync();

This style of application has been adopted into .net 6 where you then can have the following

HelloWorld.csproj (2)

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

</Project>

Program.cs (2)

var app = WebApplication.Create(args);

app.MapGet("/", () => "Hello World");

app.Run();

These templates have been published on the minimal api marketing page. Some of these improvements can also imply improvements for Giraffe.

Startup performance

Building and starting a minimal .net API in C# takes a few seconds (around 4 seconds for .net 6 and around 7 seconds for .net 5 on my machine) while starting a minimal nodejs API on Windows takes at most 1 second. These numbers should be taken with a grain of salt. Once you have thousands of lines of TypeScript the performance to parse these can impose a significant overhead.

F# has the REPL going for it so you might not have the same need for startup performance. Hot reload takes only a few seconds (maybe 2 seconds on my machine) why that makes for a nicer experience.

Conclusion

In a way the development experience when writing small web services has gotten significantly more along the lines of JavaScript. The experience is naturally different, but you can write code more in the same way.

Monorepo versus Polyrepo

webmaster@gewalli.se (Oskar Gewalli) — Fri, 05 Nov 2021 18:15:31 +0000

Let us start by defining monorepo

Keep multiple projects in a single source control system 1
Acquire as many third-party and in-house dependencies as possible for a build from the same source-control repository/branch, and in the same update/pull/sync operation. 2
Keep all teams in agreement on the versions of third-party and in-house dependencies via lock-step upgrades. 2

Plausible restrictions

The point about keeping third-party binary dependencies in source control was popular in the .net world before the NuGet and in the Java world before Maven Repositories. We have come to the point where it’s preferable to avoid this practice. Let us assume that only source code and similar sources are stored.

Language & infrastructure

Languages that do not have good package managers naturally lend themselves to creation of monorepos. The main languages that I’ve used that have this property are C and C++.

Debugging

In order to debug binary packages that you don’t have the source for some languages have support for different debug packages and source references. 3 Even if these tools are available they might not work with your organization source control or package repository.

If you write libraries for business code (spread domain knowledge in package artifacts that get referenced from an internal package repository), then having restrictions on how to debug the code can be a pain for developers.

Pain points

Matt Klein makes us aware about having tight coupling and performance in Monorepos: Please don’t!.

Tight coupling

You want to homogenize the use of internal or third party libraries. Is it OK for other teams that you refactor their code when they are busy with say a crunch and want to avoid unnecessary merge conflicts?

Does changes of internal or third party libraries imply a subtle change in behavior? Even if the code is well tested on its own, usage of the code can break. You might want to have a gradual increase in the usage of a new version of a library.

One reason why you want to transform multiple repositories to fewer repositories is that you have a monolith in disguise. Multiple services and libraries are changed for one issue in what should be one business subdomain.

Performance

For very large repositories we see that organizations use source control systems that allow you to do “sparse checkouts” of only a subset of the repository since the amount of code takes too long to clone.

Focus

Clear boundaries that you don’t need to cross can increase focus on a single business subdomain.

Being able to browse the history for the code of a business subdomain can allow people to know what’s going on (too much noise drowns out the team).

Ownership

Everybody has access to everything. Who reviews what?

Conclusion

Should you have a single repository for a team? I think it makes sense if the team only works on one product/system and the choice suits the system and the team.

Probably you want to merge and split out repositories as time goes on. It’s best to keep your options open and make a pragmatic choice for your team. Keeping separate repositories can make it clear that the source is supposed to be separate.

References

[1]

Wikipedia Monorepo

[2]

trunk based development Monorepos

[3]

Using functional programming to optimize frontend code

webmaster@gewalli.se (Oskar Gewalli) — Sun, 06 Jun 2021 09:59:41 +0000

Intro

During 2019 I started on the current project. At this project I’m working on, we are building a enterprise system for a government customer. When I got in significant parts had been developed of the frontend and backend system. My role when I started in the project was in order to help develop test automation software for the system. Since I’ve worked on both frontend and backend parts in other systems, I’m able to help out in various parts of the system.

I’m going to talk about one particular issue in the frontend code. When I got into the project, the frontend used mainly Formik for forms. There is significant complexity in the frontend code. Some of the forms are quite complicated and have a lot of behavior. While debugging we noticed that the frontend started to slow down significantly for the bigger forms. Since the end users was not affected, we hoped that it would be fine. As new requirements got implemented the slowdown during development started to manifest during end user testing.

Solutions

The question was what to do? The first obvious options are the following:

Change the design and the requirements and rewrite the existing implementation
Find a more performant library that does roughly the same thing and rewrite the existing implementation
Optimize Formik

These options are not very palatable. Changing the design and requirements would require stakeholders to agree on new requirements and a new design. Since there has been a lot of politics involved in getting the current design and requirements approved, changes would require significant amount of time.

We started evaluating different libraries. Problem with that approach is that it requires a lot of changes in the frontend code in order to evaluate if the other library would improve the performance. This could potentially end up eating a significant amount of trial and error (potentially costing weeks of effort).

While looking at the Formik code base in order to optimize it, we noticed that there is a lot of historical luggage that shows up in the code making optimizations quite difficult.

Lazy re-implementation

Since there is a lot of luggage in Formik, but we only use selected parts, we realized that given around 20 hours work we could implement a library having the API surface that we use but with better performance. Since we could evaluate different implementation options for the existing code we could do some trial and error (using the existing frontend) without having to do extensive rewrites.

During the implementation I talked to David Wincent (another developer in the team) about React performance and different approaches. We evaluated the following:

Due to the API surface that we used, it did not look as if it was suitable to use hookstate for the implementation (I had some issues getting the API to work using Hookstate). We like Redux, but we wanted to isolate state to avoid changes causing rerenders in unrelated parts of the application ().

There is a lot of code that gets executed during render instead of in the reducer in Formik. There was some insights into getting more performant code:

There are posts such as Kent C. Dodds blog on how-to use react context effectively talking about how having dispatch and state in the same React context can post impact performance.
Code executing during render impacts the user experience more than if it’s executed in the reducer
It is significantly easier to implement an existing API surface compared to writing from scratch
Existing unit tests can help speed up development by guiding development effort and lowering chances of regressions
The fastest thing to do, is to do nothing

Functional rewrite

Since React is heavily inspired by functional programming, we could apply

We could apply defunctionalization: instead of executing code in the Field array we send messages to be executed in the reducer.
Immutable data and lenses: Instead of guarding against developers not following React best practices we could assume in the implementation that the data is immutable and reuse an existing lens implementation in order to get a faster copy updates.

Conclusion

We could get a library that performed significantly better than Formik for our purposes. The time it took was around 25 hours to get to a point where the performance did not impact the end users anymore. Functional programming patterns are useful for more than the sake of correctness.

Dependency injection in FSharp

webmaster@gewalli.se (Oskar Gewalli) — Sat, 05 Sep 2020 15:11:06 +0000

One of the questions around how to write F# is how do we compose bigger business applications. In C# you generally use constructor injection.

Let us start by defining some base functions types and environment that we will use in later code samples:

open FSharpPlus
open FSharpPlus.Data

type IUserRepository =
    abstract GetUser : email : string -> Async<string>

type IShoppingListRepository =
    abstract Add : shoppingList : string list -> string list
type Env() =
    interface IUserRepository with
        member this.GetUser email =
            async { return "Sandeep"}
    interface IShoppingListRepository with
            member this.Add shoppingList =
                shoppingList

One example is to use ReaderT in to be able to provide a bag of dependencies as seen on gitter and here:

let getUser email =
    ReaderT(fun (env : #IUserRepository) -> env.GetUser email )

let addShoppingList shoppingList =
    ReaderT(fun (env : #IShoppingListRepository) -> async { return env.Add shoppingList })

let addShoppingListM email = monad {
    let! user = getUser email
    let shoppingList = ["s"]
    return! addShoppingList shoppingList
}

ReaderT.run (addShoppingListM "a@a")  (Env())
|> fun listA -> async {
    let! list = listA
    printfn "%A" list
} |> Async.Start

There is also a post about using reader monad for dependency injection but without using more advanced techniques such as monad transformers.

Another way to decompose your program is to use currying as seen on F# for fun and profit:

let getUser (repo:IUserRepository) email = repo.GetUser email

let addShoppingList (repo : IShoppingListRepository) shoppingList = async { return repo.Add shoppingList }

let addShoppingListM (getUser: string -> Async<string>) (addShoppingList:string list -> Async<string list>) email = monad {
    let! user = getUser email
    let shoppingList = ["s"]
    return! addShoppingList shoppingList
}

let composeAddShoppingListM ()=
    let env = Env()
    let getUser = getUser (env :> IUserRepository)
    let addShoppingList = addShoppingList (env :> IShoppingListRepository)
    addShoppingListM getUser addShoppingList

let addToShoppingListC = composeAddShoppingListM ()
addToShoppingListC "a@a"
|> fun listA -> async {
    let! list = listA
    printfn "%A" list
    } |> Async.Start

In the above example we reused the same class Env in order to keep the code shorter. You need to have some glue, why it might not be suitable for a solution where you have a lot of functions with a lot of dependencies.

Since F# allows for nice OO programming you can use dependency injection via constructor injection:

type Shopping (userRepository:IUserRepository, shoppingListRepository:IShoppingListRepository)=

    let getUser email = userRepository.GetUser email

    let addShoppingList shoppingList = async { return shoppingListRepository.Add shoppingList }

    member __.AddShoppingListM email = monad {
        let! user = getUser email
        let shoppingList = ["s"]
        return! addShoppingList shoppingList
    }

let env = Env()
let shopping = Shopping (env :> IUserRepository, env :> IShoppingListRepository)

shopping.AddShoppingListM "a@a"
|> fun listA -> async {
    let! list = listA
    printfn "%A" list
} |> Async.Start

I usually write F# with a mix of currying and constructor injection and have not tried out using reader monad for service dependencies. The use of ReaderT lets you remove the amount of parameters you might otherwise need to pass around. One of the neat things about using a monad transformer is that you can then compose it with your base monad that you otherwise use extensively such as async.

Implicit versus explicit dependencies

webmaster@gewalli.se (Oskar Gewalli) — Thu, 28 May 2020 16:30:50 +0000

These are musings related to the question what kind of libraries resist the test of time. Some of it is related to issues encountered due to Dependency Hell.

Explicit dependency

Let us start out by asking the question, what is an explicit dependency in your software?

How should we define a dependency? The easiest way to define a dependency would be there is an explicit package reference to a library. If you can remove the package reference, does the code still work? In order to make it clear, let us define it as a reference to a library and explicit usage of the library API.

How does such an explicit dependency look in C#:

using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Identity.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
...
    public class BusinessDbContext : IdentityDbContext<ApplicationUser>
    {
        public BusinessDbContext(DbContextOptions options) : base(options)
        {
        }

        public DbSet<Customer> Customers { get; set; }
        public DbSet<Order> Orders { get; set; }
        public DbSet<Product> Products { get; set; }
    }

The businessDbContext class is implemented using EF core. There is an explicit dependency on Entity Framework in this case.

Value of explicit dependencies

The benefit of using a specific implementation is that you can leverage the particulars of that implementation.

The cost of standardisation is the loss of agility when there is value in being able to change implementations.

Implicit dependency

In order to look for an example of a implicit dependency of code we could look at how different ORM works when using POCO or POJO. When you use a restricted amount of features from Entity Framework, your code will work with other frameworks.

Since the behavior of any ORM will necessitate that you follow the patterns and conventions assumed by the framework. For instance if you decide to use strongly typed keys, will these work in another ORM? Does your usage of inheritance work with another ORM? These small things tie subsets of your code implicitly to a specific library or framework.

Another example of implicit dependency is if you follow the advice of Mark Seeman on composition root location, you could have used say one specific dependency injection framework only in your composition root. How could that tie your other code to that then? A specific implementation of a dependency injection framework will have sometimes subtle differences in how things are created when you resolve the code. You might have unintentionally created code that works when initialized in a particular way but will break when you switch to another framework. Microsoft has used IServiceProvider as the least common denominator.

Hiccups could be:

How are concrete classes resolved, do they need to be explicitly registered?
What happens when you send an typeof(IEnumerable<T>) to GetService of IServiceProvider? Will the same thing happen for all implementations of IServiceProvider? The assumption is that it should.
What happens when you have multiple registrations of T?

if we look at the previous implementation of GetRequiredService (an extension of IServiceProvider) we note some interesting behavior:

public static object GetRequiredService(this IServiceProvider provider, Type serviceType)
{
    ...
    var requiredServiceSupportingProvider = provider as ISupportRequiredService;
    if (requiredServiceSupportingProvider != null)
    {
        return requiredServiceSupportingProvider.GetRequiredService(serviceType);
    }

    var service = provider.GetService(serviceType);
    ...

The implication is that different implementations of IServiceProvider can have different levels of compatibility with each other.

If your library can avoid a dependency injection framework and instead present a simple API for consumers of the library, it will make it more resilient against future changes. If your target audience is C# enterprise applications, then Microsoft Extensions Dependency Injection could be currently standard enough. That’s at least a decision that was made for libraries at Carable. This turned out that we needed to upgrade some libraries due to breaking changes between major versions.

We see similar implications around logging frameworks. Say that you have wrapped the logging in your own API surface in order for your library logging to work. The API surface has grown to incorporate various features needed over the years (keeping it in tandem with the logging infrastructure). Say that the organization behind the logging framework goes bust. How much work is it to adapt what is now a fairly large API surface in order to be able to still use old libraries? Since the interfaces you have defined match the features of the old logging infrastructure, you could have trouble adopting a new logging infrastructure (even though you have your own interfaces).

Fleeting standards

Any organization will take a look at the libraries they maintain and consider if it’s worth the effort of maintaining them. Microsoft is no exception. While we see Microsoft backing, some of the libraries get a huge boon. Question is what happens in 10-15 years? Will there be a new iteration of software in order to deal with the competition from other ecosystems?

I’ve seen standardizations on logging for .net come and go. In the early parts of my career Log4Net was good enough and useful enough that it made sense to standardize on that. The latest thing I’ve seen is standardizations on SeriLog and Microsoft Extensions Logging (in various mixtures).

For instance, for libraries there are a couple of logging alternatives in the .net space:

Note that for C#, there is a defacto standard in the form of Microsoft Extensions Logging. For the transition between major versions 1 and 2 there was some breaking changes. For the current iterations 2 to 3 I’ve not seen any breaking changes (around logging). Instead I’ve seen (for internal libraries at work) that for the iteration 2 to 3 EF Core has been source compatible but binary incompatible. This has meant that we had to keep the internal libraries in lockstep.

If you want to avoid having to upgrade everything at once and instead want to take it gradually, you probably want to avoid coupling your library to a major version of a common library causing dependency issues.

That’s why preference when it comes to for example library logging to choose the amount of dependencies you feel that you can handle. If you can get away with something like the following for logging:

using LogError = Action<string, Exception, object[]>;
using LogDebug = Action<string, object[]>;

You don’t really need to worry too much about different future implementations. The above definition might not be enough for your needs (why a larger abstraction is warranted).

Leaky abstractions

Leaky abstractions are somewhat related to implicit dependencies in that the abstractions still cause tight coupling to specific implementations.

Dangers

Having a lowest common interface restricts possibilities. Having defined an interface you might tie your software to a specific implementation implicitly (making you pay for the abstraction without giving you ability to swap out implementation).

Value

In order to decompose a much larger system into smaller parts in order to let people focus on subsets, having boundaries can be helpful. You might not be able to easily swap out the implementation of a complicated part of your system. Letting people ignore a lot of the implementation details can still provide value even though it might be a leaky abstraction.

TLDR

If you want to hedge your bets (in order to be able to swap out small parts) use small abstractions and small interfaces in order to avoid coupling your code to a specific implementations (when there is a reason for it). Taking as few dependencies as possible is usually the best thing for any library, since it helps making it more resilient against other libraries using different versions of the dependency.

Proxy API requests in development

webmaster@gewalli.se (Oskar Gewalli) — Thu, 07 May 2020 06:38:36 +0000

The road towards frontend and backend separation

Or at least why you should care about proxies when dealing with web applications.

Around 2005

I studied computer science at Lund university. One of the professors warned us about two things: JavaScript and Sharepoint. The web programming I got aquatinted with at university was mainly Java servlets and Python CGI apps.

I started working at MaCom in Denmark as a system developer. The amount of JavaScript needed could easily fit in a small single file. Most of the application was written in asp.net using aspx and C#. Pages written in aspx is a mix of HTML and C# code and composed as into components called “pages” and “user controls”.

There was no need for a separation of frontend and backend since most of the code executed on the server.

Around 2010

At Lundalogik I was part of a team writing a web CRM system. The amount of code in the codebase was around half C# and half JavaScript. In order to structure the code we had a large amount of JavaScript files. The server parts was developed in tandem with the client-side parts. As we expanded the team we got a frontend specialist that wanted to focus on JavaScript only. From that persons point of view it was a hassle to be forced to use Visual Studio to edit the frontend code.

If you instead work on a Windows machine you can use IIS as a proxy. Hanselman has a nice introduction. When in a Windows shop you usually go with IIS for webserver and proxy needs. Since there wasn’t any need for any proxy-configuration we used IIS only to rewrite urls.

Around this time it became clear that it’s impossible to avoid JavaScript (in some form or another). At the same time browser standardization and library adapters made JS more palatable.

Around 2017

At the startup Boolean & Brace we worked in two teams, backend and frontend. The codebase was split up into backend and frontend. The structure of the frontend was completely separated from the backend. The frontend code was structured into React components where some have routes and some components are used from other components. This mimics more the experience of developing desktop applications and was the goal of aspx (before JavaScript had matured).

Most of the front-end developers used an Azure hosted test environment. For local dev environment we used NGINX proxy to different services.

The following nginx configuration illustrates a setup where you bind frontend and backend services to port 8080 on the local machine:

upstream api-service {
  server	127.0.0.1:5000;
}
upstream frontend-service {
  server	127.0.0.1:3000;
}

server {
  listen       8080;
  server_name  localhost;


  location / {
    proxy_pass http://frontend-service;
    proxy_redirect off;
  }

  location /api {

    rewrite ^/api/?(.*) /$1 break;

    proxy_pass http://api-service;
    proxy_redirect off;
  }
}

You can avoid having NGINX installed by using docker-compose.yml:

  nginx:
    image: nginx
  volumes:
    - ./nginx.conf:/etc/nginx/nginx.conf:ro
  ports:
    - "8080:8080"

Our setup was for the front-end developers to use CORS to be able to develop against remotely hosted API. The main benefit was that they didn’t need to setup services or nginx locally.

Around 2019

At Decerno I’m again working with teams dealing with React frontend applications. The new thing I’ve seen so far has been the use of an integrated proxy in the frontend code to proxy API requests in development. This means that as a frontend developer you only need to install browser, NodeJS and an editor. The nice thing about this setup is that you also get rid of the CORS-requests you otherwise get if you have frontend and backend hosted separately.

How has it been useful

In most of the recent projects (since 2017) I’ve worked in there isn’t a mix of frontend and backend. Some of the people work as frontend specialists and are more effective when they can work in a known environment (operating system, editor, et.c.). By having a remote hosted test service you can let frontend specialists focus on client-side work. There is a lot of complexity in modern frontend development when writing JavaScript applications with modern styling making it difficult to be master of all. As can be seen you are not forced to adopt one technology to use the concept if you want your pure frontend developers to be happy.

Estimated language popularity in Sweden among companies

webmaster@gewalli.se (Oskar Gewalli) — Sun, 02 Feb 2020 16:28:01 +0000

Raw numbers

I’ve written a simple project in F# to read public information from the job agency in Sweden:

Number of listings	Language
6964	java
6085	JavaScript
5727	C#
3929	python
3359	c++
2600	c
1360	php
1230	golang
1151	TypeScript
589	matlab
442	bash
420	kotlin
381	r
344	scala
332	perl
318	ruby
220	ada
157	erlang
118	groovy
78	rust
71	cobol
63	tcl
61	F#
49	ecmascript
48	scheme
47	elixir
44	haskell
39	clojure

the numbers should be read as there has been 6960 unique listings for Java. This does not tell you if its a republished job advertisement. Most companies do not publish source code, why keyword search on job ads offer a way to give an estimate of demand.

Method

I’ve filtered out some languages since my guess is that without further analysis they are overrepresented due to the fact that they share their name with common words. Such a language is for instance “swift” and “go”. In order to give an estimation of the popularity of golang, I’ve done a crude match of the context that the word Go shows up in. My guess have been that some constructs such as:

“språket Go”
“Go lang”
“Go and “
” in Go “
”, Go,”
”, Go.”
” on Go “
” i Go “
”, Go or”
” som Go,”
“• Go”

indicate a mention of the language Go. We can estimate how popular Go is.

I’ve done some adjustments for C# since there are a some aliases such as “csharp” or “c #”.

You can find the source code for the analysis on github.

Most of the analysis is keyword matching in text corpus.

High level analysis

Most of the top languages are not surprising. The language that surprised me was Python, since it is considered niche in the subset of companies that I’ve worked in.

In the mainstream category of languages we see that Java, JavaScript and C# dominate. This should not be surprising since these are used to write business applications and web applications.

Python is roughly 1/2 as popular as Java or around 2/3 of C#. Python is used for such things as web programming, data analysis and machine learning.

C++ is roughly 1/2 as popular as Java as well. I’ve heard from people working in the financial industry using C++. I’ve also heard that some games are written in C++ (besides such examples as Doom 3). C and C++ are used for embedded application development.

One of the interesting things found in this list is that PHP seems to be still somewhat prevalent on the market but waning if you compare to for instance Python.

Typescript is used more widely than I would have assumed. At work we use Typescript in order to make Javascript programming easier when dealing with larger code bases. I’ve seen some confusion around what is runtime types and compile time types when it comes to Typescript programmers. Though Javascript seems to confuse programmers in many different ways. My personal opinion is that you should combine Javascript with heavy use of linters.

Golang seems to have risen sharply in the Swedish marketplace. Talking with a programmer focused on embedded development, I’ve heard that it’s a nice language for junior embedded developers. Since Golang is very similar to other C style languages it’s probably not hard introduce developers to the language.

Mainstream languages

If we look at the broad categories of mainstream languages we have a main category that seems popular: C style languages:

C
C++
Java
JavaScript
C#

The reason for this seems to be more align with C being popular, then C style syntax being adopted due to that fact.

Python is the odd duck in this context as it’s not C style. There is some research that suggests that it is better for novices than C style syntax.

Niche languages

Some interesting groups of languages are:

Lisps such as Scheme and Clojure
ML style languages such as Haskell, F# and OCaml
Logic programming such as Prolog and MiniKanren.

Perhaps we will see some languages such as Rust and ReasonML, where you have some ML style type-system features together with C style syntax, gain popularity? Java was heavily marketed and has huge market adoption. C# has a strong corporate sponsor. My guess is that neither of these languages will be usurped in the medium term (5-10 years). Mozilla and Facebook have not marketed Rust or ReasonML heavily in order to further widespread adoption.

Conclusion

New programmers looking for entry level jobs will probably find ample opportunities in mainstream languages such as Java/C#, Javascript and Python. Knowing C style languages seems to be a sure bet short term and long term.

Validations in F# and C#

webmaster@gewalli.se (Oskar Gewalli) — Tue, 28 Jan 2020 07:21:31 +0000

In order to give a nicer developer experience composing validation messages there are broad categories of libraries that either work well in a Java/C# style setting or work well when composing functions.

Attribute based validation libraries

The default style that can be seen in many java and C# projects is the attribute based validation approach.

public class Person
{
  [MinLength(1,ErrorMessage ="NameBetween1And50"), MaxLength(50,ErrorMessage ="NameBetween1And50")]
  public string Name{get;set;}
  [EmailAddress(ErrorMessage ="EmailMustContainAtChar")]
  public string Email{get;set;}
  [Range(0,120,ErrorMessage = "AgeBetween0and120")]
  public int Age {get;set;}
}

it offers some nice advantages. Many frameworks integrate with these style of validations. You are somewhat limited in how you can apply validations, but it can be good enough.

In F# it looks the following way:

type Person = {
  [<MinLength(1,ErrorMessage ="NameBetween1And50"); MaxLength(50,ErrorMessage ="NameBetween1And50")>]Name : string
  [<EmailAddress(ErrorMessage ="EmailMustContainAtChar")>]Email : string
  [<Range(0,120,ErrorMessage = "AgeBetween0and120")>]Age : int
}

The main downside of this approach is that it is somewhat rudimentary and can require you to write somewhat verbose custom attributes for custom validations.

Fluent configuration based libraries

We notice that using FluentValidation we can separate the validation from the type declaration:

public class PersonValidator : AbstractValidator<Person>
{
  public PersonValidator()
  {
    RuleFor(n => n.Name)
      .MinimumLength(1).WithErrorCode("NameBetween1And50")
      .MaximumLength(50).WithErrorCode("NameBetween1And50");
    RuleFor(n => n.Email).EmailAddress().WithErrorCode("EmailMustContainAtChar");
    RuleFor(n => n.Age).InclusiveBetween(0, 120).WithErrorCode("AgeBetween0and120");
    RuleForEach(n => n.Bookings).SetValidator(new BookingValidator());
  }
}
public class BookingValidator : AbstractValidator<Booking>
{
  public BookingValidator()
  {
    RuleFor(n => n.Description)
      .MinimumLength(1).WithErrorCode("DescriptionBetween1And50")
      .MaximumLength(50).WithErrorCode("DescriptionBetween1And50");
  }
}

In F# this style of validations could look the following:

[<AutoOpen>]
module ValidationsMod=
  type IRuleBuilder<'T,'Property> with
    member __.__ = ()
module Person=

  type Person = { name : String
                  email : String
                  age : int }

  type PersonValidator()=
    inherit AbstractValidator<Person>()
    do
      base.RuleFor(fun n -> n.name)
          .MinimumLength(1).WithErrorCode("NameBetween1And50")
          .MaximumLength(50).WithErrorCode("NameBetween1And50")
          .__
      base.RuleFor(fun n -> n.email)
          .EmailAddress().WithErrorCode("EmailMustContainAtChar")
          .__
      base.RuleFor(fun n -> n.age)
          .InclusiveBetween(0,120).WithErrorCode("AgeBetween0and120")
          .__

We add a custom property __ in order to end the fluent chain.

The fluent validations approach seems like a OK way to express your validations in C#, it’s somewhat less elegant in F#, but looks OK.

Computation Expression based configuration builders

The fluent style of building validators is somewhat clunky in F#, why you might want to wrap some of it in computation expression builders as can be seen in AccidentalFish:

let containsAt (s:string)= s.Contains("@")
let validateEmail (property:string) (value:string)=
  if (containsAt value)
  then Ok
  else Errors([{ message="EmailMustContainAtChar";errorCode="EmailMustContainAtChar"; property=property }])
let validatePerson = createValidatorFor<Person>() {
  validate (fun r -> r.age) [
    isGreaterThanOrEqualTo 0
    isLessThanOrEqualTo 120
  ]
  validate (fun r -> r.name) [
    isNotEmpty
    hasMaxLengthOf 128
    hasMinLengthOf 1
  ]
  validate (fun r -> r.email) [validateEmail]
}

You could also define a CE builder for FluentValidator (though that we leave as an exercise to the reader). We could see the CE approach as an F# specific alternative to using fluent builder pattern.

Applicative composition based libraries

Using applicative composition based libraries such as F#+ you are not limited to the API limitations above. You can have a strict type for validation errors that composes well.


type Name = { unName : String }
with static member create s={unName=s}
type Email = { unEmail : String }
with static member create s={unEmail=s}
type Age = { unAge : int }
with static member create i={unAge=i}

type Person = { name : Name
                email : Email
                age : Age }
with static member create name email age={name=name;email=email;age=age }


type Error =
  | NameBetween1And50
  | EmailMustContainAtChar
  | AgeBetween0and120

let mkName s =
  let l = length s
  if (l >= 1 && l <= 50)
  then Success <| Name.create s
  else Failure  [ NameBetween1And50 ]

let mkEmail s =
  if String.contains '@' s
  then Success <| Email.create s
  else Failure [ EmailMustContainAtChar ]

let mkAge a =
  if (a >= 0 && a <= 120)
  then Success <| Age.create a
  else Failure [ AgeBetween0and120 ]

let mkPerson pName pEmail pAge =
  Person.create
  <!> mkName pName
  <*> mkEmail pEmail
  <*> mkAge pAge

The main point comes from the mkPerson function that composes the different validation functions into a function that either returns Success of Person or Failure of Error list.

The main downside of this approach is that it requires you to accept that you can glue functions together in this way. Since many validation libraries come prepared with some basic validations for strings and numbers, you might also want to define a collection of validation helper functions that suits your needs.

This approach allows you to strictly define the kind of validation errors that a given part of your code can return. This has some really nice benefits when you want to communicate these to another team.

Conclusion

Validations can be expressed in different ways. There are a few main ways of encoding the validators. For F# we see an additional way of composing functions instead of using builder patterns or gluing attributes to the types that you want to validate. My impression is that it’s not a clear choice unless you add preferences to the mix.

Immutable classes in C#

webmaster@gewalli.se (Oskar Gewalli) — Sun, 26 Jan 2020 16:42:35 +0000

Problems

How do you create an instance of a given readonly class?
How do you get a new instance but with given fields with different values?

Writing readonly record classes 2007

Around 2007 I wrestled with writing immutable classes in C#. One of the key pain-points was that the constructor parameters for a class representing a database row for a specific table ended up with around 68 parameters.

Note that we already had code generation of SQL Reader row deserialization based on SQL information tables.

This was before named parameters. In order to get around these limitations I wrote a small tool that used reflection in order to generate update expressions that could be copied into code (without worrying that a parameter was wrong). Once named parameters showed up, some of the pain went away.

The main benefit of having immutable classes that could not be modified was that it made the code base easier to reason about. This was a nice feature since the code represented many different aspects of managing, scheduling, teaching and attending schools in Denmark, i.e. there was a lot of code.

Another benefit is that immutable classes are safe to pass around in a multithreaded application.

Helper tools around 2013

In order to simplify working with readonly classes I made a library around 2013 named with. Another approach explored by Andrew Arnott was to create code generator Immutable object graph.

You could also implement a t4 template in order to generate code.

Both of these approaches offered pros and cons. Main con of with was performance (though for I/O bound tasks, it’s not going to be noticeable). The main downside of the code generator was finicky to use.

Since libraries such as NHibernate supports immutable records, you could benefit from modelling readonly objects.

It was also fairly obvious that it was much easier to adopt a language that supports these concepts well out of the box (you might not be surprised that the somewhat mainstream language of choice then is F#). T4 templates where restricted to Windows, making some approaches non-portable.

There are more options now (i.e. around 2019)

During 2019 I noticed how the implementation of many dependency injection containers where using System.Linq.Expression and compile expression into method. The previous fast implementations of DI containers used IL-emit. IL-emit does not seem to offer the same benefit as it did earlier. Using expression compile you can get the performance on par with hand written code.

Immutable Object Graph is still around and have merged pull requests.

The new alternative can be found in LanguageExt CodeGen. The main downside or upside of this collection of libraries is that it is very non idiomatic C#.

We see that you get immutable objects generated from autorest and OpenAPI generator.

If you specify

"readOnly":true

Autorest generates a constructor that takes the parameters:

public partial class TodoItem
{
    /// <summary>
    /// Initializes a new instance of the TodoItem class.
    /// </summary>
    public TodoItem()
    {
        CustomInit();
    }

    /// <summary>
    /// Initializes a new instance of the TodoItem class.
    /// </summary>
    public TodoItem(double id, string text)
    {
        Id = id;
        Text = text;
        CustomInit();
    }

    /// <summary>
    /// An initialization method that performs custom operations like setting defaults
    /// </summary>
    partial void CustomInit();

    /// <summary>
    /// </summary>
    [JsonProperty(PropertyName = "id")]
    public double Id { get; private set; }

    /// <summary>
    /// </summary>
    [JsonProperty(PropertyName = "text")]
    public string Text { get; private set; }
...

while openapi-tools generates an empty constructor:

/// <summary>
/// TodoItem
/// </summary>
[DataContract]
public partial class TodoItem :  IEquatable<TodoItem>, IValidatableObject
{
    /// <summary>
    /// Initializes a new instance of the <see cref="TodoItem" /> class.
    /// </summary>
    [JsonConstructorAttribute]
    public TodoItem()
    {
    }

    /// <summary>
    /// Gets or Sets Id
    /// </summary>
    [DataMember(Name="id", EmitDefaultValue=false)]
    public decimal Id { get; private set; }

    /// <summary>
    /// Gets or Sets Text
    /// </summary>
    [DataMember(Name="text", EmitDefaultValue=false)]
    public string Text { get; private set; }
    ...

We notice that the classes are declared partial.

This could then be combined with any of the above libraries or a t4 template. These templates now works outside of Windows as well.

Microservices

We have seen a lot of interest around microservices style architectural patterns. Given a smaller size of a service, you have less need for immutable classes.

Conclusion

Writing immutable classes have gotten better support in C# over the years. We see that you now have several options to pick from (without restrictions on operating system or performance).

Working on dotnet core on Mac OS X part 3

webmaster@gewalli.se (Oskar Gewalli) — Sat, 28 Dec 2019 17:29:08 +0000

After writing the previous posts about getting some of the infrastructure working for non Windows systems, I’ve seen better ways of getting it to work. Instead of using a netfx.props file, you can use a Directory.Build.props with platform specific configuration as seen in for instance FSharp.Compiler.Service:

<Project>

  <!-- mono -->
  <PropertyGroup Condition="'$(OS)' == 'Unix'">
    <MonoRoot Condition="'$(MonoRoot)' == '' and $([MSBuild]::IsOsPlatform('Linux'))">/usr</MonoRoot>
    <MonoRoot Condition="'$(MonoRoot)' == '' and $([MSBuild]::IsOsPlatform('OSX'))">/Library/Frameworks/Mono.framework/Versions/Current</MonoRoot>
    <MonoLibFolder>$(MonoRoot)/lib/mono</MonoLibFolder>
    <MonoPackaging Condition="$(TargetFramework.StartsWith('net4'))">true</MonoPackaging>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net45'">$(MonoLibFolder)/4.5-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net451'">$(MonoLibFolder)/4.5.1-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net452'">$(MonoLibFolder)/4.5.2-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net46'">$(MonoLibFolder)/4.6-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net461'">$(MonoLibFolder)/4.6.1-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net462'">$(MonoLibFolder)/4.6.2-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net47'">$(MonoLibFolder)/4.7-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net471'">$(MonoLibFolder)/4.7.1-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net472'">$(MonoLibFolder)/4.7.2-api</FrameworkPathOverride>
  </PropertyGroup>

</Project>

Another stumble has been that the nuget version on Ubuntu is a bit old. There are some ways around this, one easy way is to include a NuGet.config with both v2 and v3 feeds:

<configuration>
 <packageSources>
    <!--To inherit the global NuGet package sources remove the <clear/> line below -->
    <clear />
    <add key="nuget" value="https://api.nuget.org/v3/index.json" />
    <add key="nuget v2" value="https://www.nuget.org/api/v2" />
 </packageSources>
</configuration>

Commands and services

webmaster@gewalli.se (Oskar Gewalli) — Sat, 28 Dec 2019 09:13:25 +0000

Command handlers are services

If we start from the command redux post by Ayende. In actual shopping scenarios we need to deal with not a clear script of actions that can be executed in a sequential manner. Instead we have a unidirectional wizard-style flow where the customer is redirected from the shopping site to payment provider and back. Let us ignore these complexities and look at the following:

  var placeOrderCommand = new CompositeCommand(
    new RegisterOrderCommand(),
    new ReserveStockCommand(),
    new ChargeCardCommand(),
    new ShipOrderCommand()
  )
  {
    Sequential = true,
    StopOnError = true
  };

we note that the limitations around using simple compositions of commands. The beauty of the command pattern is more obvious when the entire user action can be encapsulated into a class such as a service class in DDD could also be called a command handler.

  public class PlaceOrderService
  {
    private readonly BusinessDbContext _dbcontext;
    ...
    private readonly ProductSettings _productSettings;
    public PlaceOrderService(
      BusinessDbContext dbcontext,
      ...
      IOption<ProductSettings> productSettings)
    {
      _dbcontext = dbcontext;
      ...
      _productSettings = productSettings.Value ?? throw new ArgumentNullException(nameof(productSettings));
    }

    public Task Execute(PlaceOrderCommand command)
    {
      // register order in DB
      _dbContext.RegisteredOrders.Add(new RegisteredOrder{ ... });
      ...

      await _dbcontext.SaveChangesAsync();
      ...
    }
  }

Reference:

Since dbcontext in asp.net core Entity Framework implements unit of work the command handler uses only one unit of work context. If you use a sql connection you could instead replicate this behavior with a transaction.

You can then compose your code as following:

flowchart TD

Http[Controller/Page/Http handler] --> C[Command handler/Service]
C --> Repository

Read more about command handler in the following README.

Separation of database and domain logic

Even though it is good design to keep your domain logic ignorant of persistence, dependency on ORM or a specific database concepts can create an implicit dependency on the persistence technology that you choose. In many simple scenarios it doesn’t make sense to isolate persistence completely from the domain model. In Ruby on Rails the choice is to spin up a sqlite database for small application tests. The same can be done with many modern ORM.

An alternative approach is to have a database with the same schema available for testing and then either letting the database become “dirty” or put the entire test under a transaction that is rolled back so that the database is kept clean between tests.

An alternative view is to avoid using an ORM and instead using a simpler mapper in order to put more of the domain logic in the database.

flowchart TD

Http[Controller/Page/Http handler] --> C[Command handler/Service]
C --> SQL

You could even skip a layer and let the controller talk SQL directly if the service abstraction is not needed. You can still design your database such that stored procedures match what would otherwise be command handlers.

flowchart TD

Http[Controller/Page/Http handler] --> S[Stored procedure]
S --> SQL

The downside is that such a scheme complicates unit testing. If you have simple domain logic that seldom change, you might less need for an extensive test suite.

Separation of bounded contexts

If you want to go further along the line of modular monolith or microservices you want to decompose parts such that:

flowchart TD

Http[?] --> D_A[Domain Service in A]
D_A --> R_A[Repository in A]
D_A --> M[Mediator]
M --> D_B[Domain Service in B]
D_B --> R_B[Repository in B]

In the modular monolith scenario A and B are seen as separate parts but are deployed at the same time.

There is a cost/benefit of introducing loose coupling. Why a nicer approach could be to:

flowchart TD

Http[?] --> D_A[Domain Service in A]
D_A --> R_A[Repository in A]
D_A --> M[named interface]
M --> D_B[Domain Service in B]
D_B --> R_B[Repository in B]

The cost of having a mediator pattern is that it makes it harder to follow the system logic.

Conclusion

Domain services map well to command handlers. Embracing SQL can be a fruitful approach. You need to evaluate the need for separation or abstraction as you go.

Configure Airport printer on Windows

webmaster@gewalli.se (Oskar Gewalli) — Sat, 21 Dec 2019 20:01:46 +0000

Setup Apple Airport printer on Windows.

Connect the printer using USB directly

Windows will auto install the needed drivers.

Setup the printer as a network printer

Reconenct the printer into the airport. Select could not find printer among the listed then start going down the printer Wizard flow:

In order to connect to the printer we add the router IP address as the “port”

Then select Custom settings:

Note that the default Port Number is correct and continue:

Make sure to use the printer driver for the printer that Windows installed after connecting the printer (Windows will install a range of printers, select the one you use):

Using multiple keyboard layouts on Windows 10

webmaster@gewalli.se (Oskar Gewalli) — Sun, 24 Nov 2019 15:01:46 +0000

Why

C style programming languages are designed in the cultural context of having a English keyboard layout. Programming C style languages on a Swedish keyboard layout is somewhat awkward since you need to exercise finger gymnastics in order to use common characters like {}\. Unless you switch to Python the easiest fix is to use multiple keyboard layouts.

How

I started out by adding English (US) and then Swedish (Sweden) however, I ended up with a long list of keyboard layouts in the keyboard selection that comes up when you press WIN+space. Among those where layouts I couldn’t figure out how they ended up there. A coworker helped me out by telling me that what I wanted is to use the English (UK) language but with US keyboard, since otherwise you will end up with additional UK keyboard layouts.

What I wanted was this configuration

but for the UK language you add the US keyboard layout that you prefer:

In my case it was the standard US keyboard layout:

UTF-8 issues in git-bash

A somewhat related issue is that you can have issues around locale in git bash. Following the instructions on stackoverflow should suffice.

Ok, so now Windows has updated and I’ve got three keyboard layouts

Note the extra keyboard layout and locale, could be for instance “English (United Kingdom)”, and add that locale and keyboard in the Language parts of settings.

Add the extra keyboard so that you end up with the extra keyboard:

Then go ahead and delete that keyboard while making sure that your currently selected keyboard is not that one.

Masstransit

webmaster@gewalli.se (Oskar Gewalli) — Mon, 02 Sep 2019 09:09:59 +0000

MassTransit is a wrapper around message queues, such as RabbitMQ,in order to simplify work on distributed applications using .NET.

What is it good for

One common task is to make sure that services react to relevant events. Instead of polling you can tie your system together using async events.

Put entity/article/object in a search engine once it’s published
Update information in data warehouse
Trigger information requests from external partners

Another usage is to handle long running tasks without the user waiting on them. Say for instance that you fire off an event or a command once an offer is requested. There might be potentially many integrations involved in a business flow, why it then makes sense to return early with a first message to the potential client “We are processing your request”, then you could send email and server side sent event to the user once you have finished processing.

Having message queue allows you decouple services. If you publish an event, there isn’t a strict need for a specific receiver. This allows you to start work on “basket” service before a “catalog” service is fully finished (then wire up logic to deal with inconsistencies later on).

MassTransit wraps common message queues and presents you with an uniform API. This is similar to what we have with entity framework for SQL.

What is it not good for

RabbitMQ et.c. throws away the messages after they have been successfully processed. If you want to be able replay old events, then this is not for you.

MassTransit (and similar wrappers) makes assumptions about how you want to deal with failures, what kind of format and how send vs publish works. This can mean that there are some discrepancies when implementing publishers and receivers in other languages than C#. Going towards a more heterogenous environment you will probably need to invest more into getting to know your message queue. In a previous project there was a need to invest in adjusting an AMQP wrapper for nodejs since the existing wrapper was out of date and did not implement the correct error handling. There is a Java port where the author mentions that he only needed a subset of features available from MassTransit in Java.

If you want to send out events on a very granular high volume, say each time someone views a page on a very popular site, then this is probably not for you. You should look at alternatives like Kafka.

Problems with distributed transactions

Transaction lock on multiple databases is a known costly affair. There are a couple of options of dealing with this. One way is to have a really large database so that the transaction is over a single database. Another way is to use the saga pattern to avoid those issues. Note for instance The Coffee Shop OrderStateMachine.

Another way is to think about data in a slightly different way: For instance by storing events and later on generate the resulting relational view of data at one point in time.

Setup

If you are on Azure then Azure Service bus is simple to setup. There are also NuGet packages to use MassTransit with AmazonSQS and ActiveMQ but these are more recent than the RabbitMQ and Azure Service bus packages. Easiest way to start locally is to use RabbitMQ by writing a docker-compose file:

version: '3.4'

services:
  rabbitmq:
    image: rabbitmq:3-management-alpine
    ports:
      - "15672:15672"
      - "5672:5672"

Cost

Masstransit is open source and free. RabbitMQ requires hosting. See AWS and Azure for pricing of their offerings.

Conclusion

MassTransit (and similar) allows you to wire up loosely coupled distributed mediator. Since MassTransit allows you to use either RabbitMQ, AWS or Azure native message queues it is fairly easy to have it up and running. It is a leaky abstraction when you are going towards a heterogenous environment (but reimplementing subsets of MassTransit can be good enough).

Is there microservices or is it an architectural dream cloud

webmaster@gewalli.se (Oskar Gewalli) — Mon, 26 Aug 2019 10:22:44 +0000

What is a microservice

There isn’t a clear definition of microservices architecture unless you pick one persons strongly held belief. This can mean that what someone considers microservices, someone else considers to be something else.

To summarize a few of the available characterizations and definitions:

James Levis and Martin Fowler

organization around business capability
automated deployment
intelligence in the endpoints, not the transport
decentralized control of languages and data

Jimmy Bogard

A microservice is a service with a design focus towards the smallest autonomous boundary.

Gartner

The characterization is that microservices are

tightly scoped
loosely coupled
control all access to their data

Sam Newman

Microservices are small, autonomous services that work together.

Smartbear

focus on building single-function modules with well-defined interfaces and operations

Rachel Reese

She mentions that there are the following guidelines around such services:

Being able to rewrite each service in fewer than six weeks
Using DDD, wherein there should be one service per bounded context
That each microservice has fewer than 300 lines of code
That each microservice is a single function

She then mentions Jet.com’s guidelines:

It’s a short script file with well-defined inputs and outputs.
It should be an application of the Single Responsibility Principle, applied at the service level.

As seen on github

Note for instance some of the styles presented in the these examples:

The future of microservices

We can expect there to be different types of standardization efforts of microservices solutions. The definition of a service and microservice has deep meaning for the type of solution you can realize. Different people will sell their view of what it is and tell you that “you are doing it wrong” in order to sell consulting/tools/hosting et.c..

Towards Angular vs React

Some sources have bigger influence than others in certain contexts, that means that different definitions can be considered decided more correct depending on the company.

Thus one potential possibility is that some organizations decides on bunch of patterns and tools that then becomes extremely popular. Compare with what has happened on the frontend side with Angular and React providing a ready made solution that tells you how you should build your app.

Kubernetes seems to have made a huge impact on the microservices world and hosting providers. The big question is if we will see any big contenders in the future.

Towards modular monolith

Lets define it as a monolith composed of modular parts in order to structure it in a way to make development clearer. We will probably see more work towards this style of architecture due to pain around one way of defining “microservices”.

See for instance kgrzybek modular monolith

Towards functionservices

Instead of going towards bigger things we will probably also see a move towards smaller services. Some of the definitions above hint at a microservices view that is more around writing very small things that resemble functions instead of being built around a bounded context. Perhaps serverless computing solutions will help solidify this style.

What problems does microservices solve

Note the focus on the following keywords: decentralized, autonomous, loosely coupled, single responsibility.

Splitting a system into separate parts can help with making it easier for developers to reason about smaller parts (given that they are sufficiently independent). It’s easier to have semantic alignment in a smaller corpus.

It’s harder to keep a big system consistent with new patterns compared to keeping smaller parts consistent locally. A heterogenous solution can thus imply easier local upgrades.

What problems does microservices not solve

How to split a system into components
Scaling or uptime issues
Spaghetti code
Sliced toast bread

In order to solve these problems you need to architect your system[s] in such a way that these issues are feasible to address.

Comparison with SOA

I’ve heard people reject some implementations as SOA. This does not make sense since you need to qualify both microservices and SOA and then end up in a straw man argument.

Note that some SOA vs microservices presentations can be used to sell a migration project from an existing SOA implementation.

Making it clearer

Since there is no common definition of microservices, you need to define what you mean. One way to make it clearer is to offer one of the known definitions. Another way is to pick parts from others and create your own.

Conclusion

Expect further confusion around the concept as previously seen around SOA. We will probably see the emergence of “modular monolith” or other architectural patterns that are defined in opposition of some definition of “microservices”. Some tools and patterns might become very popular creating a few competing solutions.

Why do we need microservices

webmaster@gewalli.se (Oskar Gewalli) — Tue, 20 Aug 2019 08:08:49 +0000

Polyglot

I’ve found that I’m way more productive in some languages compared to others. This is common enough occurrence according to ThoughtWorks that they mention polyglot programming under their tech radar in techniques. It also allows the use of heterogenous teams that can bring different strengths to the table. Sometimes a language/virtual machine has a library or framework especially suited for your problem/task.

Code chunks

Shitty code behind nice interfaces

If we look at the group intercommunication formula: n(n − 1) / 2, it should be fairly obvious that having tight coupling between teams without clear interfaces causes increased communication need. By defining nice API with nice enough documentation to communicate between team responsibilities you can ensure that teams can focus. I mean focus in the sense that team members are not led astray by leaky abstractions by exposing internal implementation details causing confusion for the uninitiated. As seen in the previous post, we do not strictly need to use microservices in order to satisfy this goal.

One of the unintuitive facts of programming is that by restricting the possibilities can lead to improvements. This fact can be seen when it comes to static type systems. The same can be applied to API evolution. By making it harder to evolve an APIs, we can make it easier to be backwards compatible. Having a more focused API for a subdomain also makes it easier to reason about cache-logic and scaling. By keeping API’s backwards compatible you can evolve separate parts of your system at their own pace.

Cost and planning

Measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs

Software estimation techniques show that the time and cost of software is closely related to the line count of the system where changes are made (see Software Estimation). This implies that having subsystems that can be reasoned about in isolation means that it’s easier to estimate and plan changes (as well as keep cost reasonable).

The cost of microservices has in fact decreased over the years. Note for instance that deploying a hosted Kubernetes solution with a light workload can cost you as little as around 30$ per months depending on your vendor. Deploying multiple sites into a single App Service on Azure can be quite cheap as well.

Conclusion

Microservices allows you to build small isolated services in different languages accessible by HTTP requests. The operational cost gives you flexibility. The difficulty and cost of hosting microservices has become more democratized, why it has become more feasible for many workloads.

Why we don't need microservices

webmaster@gewalli.se (Oskar Gewalli) — Sat, 17 Aug 2019 08:02:01 +0000

Single deployment

If the system you are developing is intended to be delivered by a single team (or a single backend team), there might be less need for a decomposed solution where parts can be deployed separately. This also applies if you are starting out on something relatively small and don’t have the operations setup for doing microservices.

If you still will want to structure your code in separate parts with clear API boundaries between them you can define request/response, publish:

    /// <summary>
    /// Defines a mediator to encapsulate request/response and publishing interaction patterns
    /// </summary>
    public interface IMediator
    {
        /// <summary>
        /// Asynchronously send a request to a single handler
        /// </summary>
        /// <typeparam name="TResponse">Response type</typeparam>
        /// <param name="request">Request object</param>
        /// <param name="cancellationToken">Optional cancellation token</param>
        /// <returns>A task that represents the send operation. The task result contains the handler response</returns>
        Task<TResponse> Send<TResponse>(IRequest<TResponse> request, CancellationToken cancellationToken = default);

        /// <summary>
        /// Asynchronously send a notification to multiple handlers
        /// </summary>
        /// <param name="notification">Notification object</param>
        /// <param name="cancellationToken">Optional cancellation token</param>
        /// <returns>A task that represents the publish operation.</returns>
        Task Publish<TNotification>(TNotification notification, CancellationToken cancellationToken = default)
            where TNotification : INotification;
    }

The above interface is from the MediatR framework, but should be fairly trivial to implement on your own.

Plugins

How would you wire up such a solution? My instinct would be to use plugin architecture to be able to load the different parts into a main project:

In for instance Jenkins you have the ability to write plugins.
MEF is used by Lucene.net in order to create docs plugins
CMS systems usually support plugins in order to extend the functionality. See Wordpress plugins or Sharepoint add-in.

If we contrast microservices to plugins what do we find?

Plugins usually need to follow the package versions and language restrictions of the host runtime.

There isn’t always a natural way to test plugins in isolation, while for a http service you can use curl.

There is an operations overhead for using several deployments (though that can be negligible depending on your solution).

Modularized monolith

Another approach is to avoid the plugin style and create a repository with sub-folders for separate modules, then a single delivery project that aggregates all of the modules. You still have questions also seen for plugins. One benefit is that you avoid needing to define extensibility points in order to allow plugin development.

One example of this approach can be found in SimplCommerce. Note the use of both frontend and backend parts per module.

Serverless

As we see on Martin Fowler this type of solution can be thought of as a style of microservices solution.

We see that there is vendor specific solutions and open source versions in this space. The most interesting things I’ve seen here are open source implementations of serverless on top of Kubernetes.

By going serverless you can avoid some of the operations aspects of microservices (especially if you go with a cloud provider solution).

Conclusion

There are some pros and cons to going with one architecture over another. Depending on what you build, it doesn’t always make sense to go with a microservices approach why the other styles might be applicable.

Why do we need message queues?

webmaster@gewalli.se (Oskar Gewalli) — Fri, 02 Aug 2019 08:06:01 +0000

HTTP might be good enough

For many systems there isn’t a need for message queues. Regular HTTP-style request/response are often enough. If your system is architected without thinking about events you might have very little need for it.

A webhook can be good enough. It all depends on your architecture and business scenarios if it makes sense.

What good is a message queue

A message queue is good for dealing with asynchronous messages not expected to be dealt with immediately. You can also use a message queue to create loosely coupled systems.

PUB/SUB

One of the neat things with having a properly architected publish/subscribe is that you can decouple parts of your system. You can for instance use message systems with routing in order to have many consumers for selected messages or events. Products that fit the bill are:

RabbitMQ
Azure Service bus
ActiveMQ

ZeroMQ looks particularly interesting as it doesn’t require a dedicated deployment. Though I’ve never used it.

The assumption is that some systems are not interested in receiving some events. We see that eShopOnContainers can use both RabbitMQ and Azure Service bus.

I’ve found that RabbitMQ style communication can be useful for dealing with longer running tasks generated by user input (not to be confused by cron-jobs). It’s a known type of technology that has been around. Note for instance some of the standardization around AMQP.

Event stream

If pushing a huge number of messages/events then routing might be less of a concern why you might want to look into systems like:

Kafka
AWS Kinesis
Azure Event Hub

We have an example of using Kafka from RedHat. We have also projects from Jet using Kafka: Note the mention of Kafka for projection in Equinox. Kafka is also used by Netflix and Spotify.

Kafka has gotten a lot of attention in recent years and is probably still considered hot to work with.

Conclusion

We might not need message queues for all systems and services, but they can be useful tools for certain use cases. You also need to decide on what style of message queues you need for your system.

ESB and service mesh

webmaster@gewalli.se (Oskar Gewalli) — Sun, 28 Jul 2019 18:58:14 +0000

Service mesh

Service mesh looks like an evolution of Enterprise Service Bus and has similar focus. An ESB is a single point of failure. Since it’s a combination of many concerns, it can be hard to get right. A service mesh tries to mitigate this by focus mainly on HTTP, HTTP2 style communication and being distributed as a deployment (each service has it’s own copy of the mesh software).

Typical concerns

Security (restrict traffic between services, use ssl for internal communication)
Observability (being able to trace requests and communication between services)
Traffic shaping (route traffic to avoid faulty services)

When does it make sense

A service mesh starts to look really attractive once you start get into something like hundreds of services written by several teams. For fewer services it might introduce unnecessary operations overhead. This should be similar to recommendations around ESB. This should also be depending on if your service provider makes it easy to enable such a service. Using a service mesh also looks more attractive than using ESB since some of the issues are mitigated.

When doesn’t it make sense

If you have mostly things hosted on one type of web server/service host, then collection of web server logging might be enough to satisfy the observability requirement. For instance when your hosting provider makes it easy to enable HTTP log collection. Going towards more dockerized services shifts your needs (as you do not have as homogeneous environment).

The security requirement could also be satisfied by decent middleware. Many HTTP frameworks have JWT support. Your hosting provider could let you enable let’s encrypt for sites.

Choose your own adventure in F# 2

webmaster@gewalli.se (Oskar Gewalli) — Sun, 28 Jul 2019 13:56:23 +0000

In the first post I introduced some alternatives for F# programming. The title alludes at classic roleplaying games. Mattias Brandewinder has written a blog post series about writing DnD logic in F# that should be an interesting read in order to follow up on that lead.

Maven security issues scanning

webmaster@gewalli.se (Oskar Gewalli) — Sat, 27 Jul 2019 05:20:08 +0000

There is a mature package management system in the Java world. Maven seems to be quite standard. I’ve made a sample project with Hibernate and Spring. One of the neat things I found out was that it’s quite easy to integrate OWASP checks. Following the guide to setup a simple way to be able to that your closed source system does not have any known vulnerabilities in dependencies.

Decompiling Java binaries

webmaster@gewalli.se (Oskar Gewalli) — Fri, 26 Jul 2019 19:02:44 +0000

There are some tools out there to decompile Java binaries. I’ve found that the easiest one to process an entire jar is Fernflow.

In order to build you notice that it’s built using Gradle. Either install gradle or use existing shell wrappers to build the jar file.

Once you have built the decompiler you might want to wrap it up with a shell wrapper and put the executable in your path next to the fernflower.jar:

#!/usr/bin/env bash

java -jar `dirname "${0}"`/fernflower.jar $@

Java vs C#

webmaster@gewalli.se (Oskar Gewalli) — Sat, 15 Jun 2019 17:37:10 +0000

Java started out as a very object oriented language. One of the nice things with Java (as a language) has been that the syntax has been quite stable for a long time. C# competes with Java for mainstream developer mind-share. C# has pushed better tooling and more features but was primarily focused on Windows.

For the last few years we have seen .net becoming a truer cross platform language with the .net core project and Microsoft’s acquisition of Xamarin. Due to the push towards micro services and Docker this makes a lot of sense since we will probably see more large scale enterprise software use some sort of micro service architecture to allow scaling of development teams.

Features

Implemented in both

Language Features

Some of the features we know and love in C# is nowadays also part of modern Java:

The ability to create a small lambda simplifies many programming tasks why the functional interfaces in Java 8 is a nice addition.
Many of the bread and butter collection operations seen as extension methods in .net are available by using java.util.stream
There is local variable type inference with var in Java 10
In more modern Java you can write try (T instance=new ExpensiveResource()){ ... } to automatically dispose an instance that implements the AutoCloseable interface.

In the C# corner we are waiting on C# 8 for:

Default interface implementation that we have seen used in Java since Java 8.

We see that Java uses standard generic class Optional<> with methods while C# uses ?. and ?? operators and uses Nullable<> for value types.

Tooling

Using Visual Studio Code to develop Java feels very productive. Using Jetbrains Intellij is nicer. I’m not a fan of Eclipse. For C# I usually prefer Visual Studio while Visual Studio Code works and you get some more sweet refactoring in Rider (or through Resharper).

Both Maven and NuGet are mature and enable decomposition into libraries.

Missing features added through extension

The missing part that becomes quite obvious as you start working on any project in Java is the lack of auto properties. However, there is Project Lombok that removes a lot of boilerplate from your code.

You would expect that in C# you’d use t4 template (or Roslyn Source generators) to reduce boilerplate.

Missing features

There isn’t any generic support in the Java runtime. This leads to some weirdness around type erasure.

There isn’t any anonymous interface implementation in C#, but there is in Java (and something similar in F#).

Style

Differences

The style of stream and the usage of words are more in line with many other languages (see for instance Javascript, Ruby, Python). Notice the use of map, filter and reduce while in C# LINQ is more in line with SQL. This gives the code a slightly different flavor.

Checked exceptions cause some boilerplate code that becomes a bit more manageable with Result style error handling.

Similarities

There is validation in Java similar to the familiar attribute based validation in .net.

There are popular MVC frameworks for both. Spring should feel familiar to anyone using asp.net MVC.

There are some gotchas, but any .net developer or Java developer should be able to work in both. There might be a short term productivity loss.

Conclusion

Both languages have mature editor support. Many of the bread and butter programming tasks have feature support in both languages. There are some stylistic differences between the languages.

When is Go a nice language?

webmaster@gewalli.se (Oskar Gewalli) — Thu, 30 May 2019 10:15:09 +0000

I’ve been following Go for a while due to the influence of one of my personal heroes: Ken Thompson (due to his hand in making Unix and C programming language).

Many years ago I met Douglas Crockford. At that point I was looking into low level implementation of parsers why I suggested a JVM language with more primitive structures in order to simplify implementation of more low level performance critical parts:

If you make a JVM language with goto people will start coding web apps in it.

The comment was in a way prophetic since that’s sort of what has happened with Go. In the sense that people started using what is essentially a low level language to write web applications.

The good

Very procedural

Go fits procedural programmers naturally: Since many programmers start out thinking procedurally and debugging code in order to understand it, it should come naturally to many developers.

There are few conceptual components of Go. Like C, it’s not really a difficult language to learn (though there are some constructs related to GoRoutines that looks like false friends).

Simple reflection

There are rudimentary reflection capabilities, why you could implement some standard meta programming patterns found in Java/C#.

Compilation time

Go has short compilation time. This can improve developer happiness (reduce the code/compile/try cycle). It is also easy to use in non standard editors (as opposed to C#/Java).

Size of docker deliverables

Compared to some other more mature languages like C# and Java: Go produces smaller docker deliverables. Though if you really care about the size of your deliverables Go might not be the optimal language (but perhaps good enough).

If you are concerned about the size of your deliverables, then you could write web apps in c with say kore.

Code formatter

I like that Go comes bundled with a code formatter, since c languages otherwise leans towards bike shedding when it comes to formatting.

Testing

There is a nice story around testing for the language that also could improve developer productivity.

Structural types

In order to implement an interface you use the fact that Go has a structural type system.

The bad

Missing parts

One of the notable things in Go is that there are no exceptions. This is however a half truth. There are exception like functionality as can be seen on Matt Kleins post on Medium. It’s not used that often, why usage of it may be frowned upon by the designers of the language.

Conclusion

The error handling and lack of generics require you to write more code. The performance is comparable to other statically typed languages. My guess is that you can be productive enough in order to write regular business code in it. It can be used as an alternative to Java and C# for when the team likes Go-style code.

Docker and .net

webmaster@gewalli.se (Oskar Gewalli) — Wed, 22 May 2019 19:35:44 +0000

I’ve spent half a year on a project to Kubernetes project. During that time we did production deployments of .net core web services to docker.

My main suggestion would be to follow the example on docker.com.

Kubernetes and distractions

webmaster@gewalli.se (Oskar Gewalli) — Mon, 15 Apr 2019 05:47:14 +0000

There are many cool projects you can install into Kubernetes. However before buying into the hype, try to focus on the basics in order to deliver value to the business/customer.

Start with few components

You can start by following Alex Ellis guide on HowTo move your project to kubernetes.

One thing to take away from Julia Evans talk: High Reliability Infrastructure Migrations is:

ignore most kubernetes ecosystem software

Kubernetes on the cheap

webmaster@gewalli.se (Oskar Gewalli) — Tue, 09 Apr 2019 05:32:29 +0000

You can use the fact that Kubernetes allows you to get a stable environment on unstable hardware, so that you can use a bunch of cheap commodity computers to scale your deployments horizontally. A good starting point would be to build your own test cluster.

Jeff Atwood talks about using co-location in the cloud is just someone elses computer
Hanselman tells you How to Build a Kubernetes Cluster with ARM Raspberry Pi then run .NET Core on OpenFaas

Another approach is to look into where hosted Kubernetes is more affordable than say EKS.

kubeadm-aws
an example where someone used GKE to get hosted Kubernetes for personal projects

When does it make sense to use Kubernetes?

webmaster@gewalli.se (Oskar Gewalli) — Mon, 08 Apr 2019 19:14:03 +0000

The bad

There is a certain level of complexity associated with Kubernetes. Depending on how you get started, it can be a bit of a chore to get a cluster up and running. There are costs associated with having a stable cluster up and running.

The good

What does Kubernetes give you? It offers an environment for hosting docker services that is more portable between vendors. It allows you to run an environment locally mimicking your production environment. You are able to gradually roll out new versions of services and scale them according to need.

My Guess

I would guesstimate that if you have a few services (say 5-10) without a network topology and your scaling needs fits a single computer, then kubernetes is probably overkill. While if you want or plan to host a multitude of different services then the complexity looks justified.

You have a lot of service/site/job deliverables that work well as docker images
You want the infrastructure to help maintain uptime and roll out deployments
You estimate that Kubernetes will be prevalent in the future and want in on it now

Similar Alternatives

Note however that there are some other alternatives you might want to look into. See for instance Docker Swarm or Service Fabric and OpenShift.

Choose your own adventure in F#

webmaster@gewalli.se (Oskar Gewalli) — Sun, 06 Jan 2019 09:32:39 +0000

When you start out on F# your first thought might be:

let square x = x * x

let sumOfSquares n =
   [1..n]
   |> List.map square
   |> List.sum

hey, that sort of looks like:

public static class Utils
{
   public static int SumOfSquares(int n)
   {
      return Enumerable.Range(1, n)
         .Select(i => i * i)
         .Sum();
   }
}

Examples from F# for fun and profit: Sum of squares.

At a first glance, F# looks sort like slightly different style of C#, where instead of using extension methods you use static methods. Turns out that that’s not all there is to F#.

Let’s start from the basics

There is also an excellent source of knowledge about functional programming called F# for fun and profit.

Overlap of C# and F# coding

There are some points to being happy in F#. For instance there is an intersection of styles for F# and C# where both languages are nice to use:

Lightly functional style ( Select or seq.map, Where or seq.filter, pure functions, named types instead of using string, int, int64, Dictionary<string,object> et.c., lambda expressions)
Simple Object Orientated code (i.e. no deep type hierarchies)
Mutable constructs

Prefer F# over C# in the case when …

Some constructs that are easy to use in F# need more work or make for weird C#.

Immutable constructs ( F# tries to steer you into immutable by default and has constructs to help you). In C# you need something like With or Immutable Object Graph
Computation expressions (builders that can help you write composable abstractions)
Discriminated unions (in C# your best bet is to look at OneOf or use FSharpX)
Type providers (in C# it’s better to generate source code using something like t4 templates, though that can become awkward)
Statically Resolved Type Parameters (can be used to avoid using reflection)

Prefer C# over F# in the case when …

Some constructs that are easy to use in C# need more work or make for weird F#

Unsafe code, Native pointers (it’s possible in F#, but is more verbose)
Goto (generally seen as an anti-pattern but is really useful when writing highly optimized code: think core parts of asp.net MVC)
Deep type hierarchies (generally seen as an anti-pattern and in F# the language tries to steer you away from these patterns, though in certain cases they can be useful)
Code generation (due to type providers, there is less need for it in F#)
Implicit type conversions (generally seen as an anti-pattern, you can explicitly use implicit operator in F#)

There is also a certain mainstream appeal of c style languages, why you might choose to code in C# despite being fluent in F#. I’m not fluent enough in VB.net to know when to use that language.

So you have choosen F#?

Let’s say that you go start with a server or service (and worry about client or web code later).

Web framework overview

There is a smörgåsbord of web frameworks to choose from.

Suave : Oldie but goodie. Some old idiosyncrasies. Need some love and major version with breaking changes removing cruft.
Giraffe : in the style of Suave but with better performance (not as composable due to usage of Task over Async)
Freya : Using custom computation expressions as builders. Based on webmachine. Need some love and new major version removing cruft.
Frank : Recently overhauled. Frank together with Freya and Suave are the oldest F# web frameworks. Using computation expression as builder.
Saturn : Relatively new to the block. Using custom computation expressions as builders. Based on Pheonix and MVC patterns.
asp.net MVC : MVC is the most common .net run of the mill object oriented web MVC framework.
based on IApplicationBuilder or OWIN: OWIN is considered dead at this point, but it played an essential role in influencing some of the work that went into asp.net core. You can compose your API in a slightly rough style by leveraging these abstractions.

The question about web framework can also be phrased as, what kind of style do you like? Roughly there are five categories.

Function composition based : Suave, Giraffe

My personal favorite. I like being able to compose API out of function building blocks. It’s both abstract and low level at the same time. Might not be appropriate for larger teams, but fits well with microservices.

Giraffe is bundled with Newtonsoft.Json for serialization. Suave uses DataContractJsonSerializer for serialization. Both of the libraries make it very easy to plug in your own choice of Json framework.

Personally I prefer to plug in Fleece with FSharp.Data.

Bring your own abstractions: IApplicationBuilder

Another one of my favorites. You can create your own abstractions on top of Map, MapWhen and Run. You can also choose to be more to the metal in order to avoid abstractions. This can be excellent in order to teach programmers about how the details of web services (or when there is little need of abstractions).

Webmachine based : Freya

Focus here is on being true to the HTTP spec. The webmachine diagram is used by people to teach themselves about HTTP.

The Freya tutorial points you in the direction of Chiron (from the same github organization).

CE as builder[s] : Frank, Saturn

Frank is a kind of small library that is probably good enough for many use cases. Saturn strives to be a batteries included framework.

Object oriented : ASP.NET MVC

Even though you need some attributes to configure endpoints, you can get a lot of mileage from TaskBuilder.fs and the fact that you need less code to take in dependency injection parameters. F# shows that it can be a nice object oriented language.

The default for asp.net mvc is to use Newtonsoft.Json. It’s not hard to plug in another serializer.

What about testing your code?

Main branches

There are two main paths of F# testing (that I’ve seen). One is the *Unit style testing frameworks, the other is composable testing frameworks.

*Unit

NUnit classic unit testing framework for .net.
XUnit is rearchitected testing framework by some of the original team from NUnit made to make unit testing more accessible and safe (i.e. trying to steer away from anti-patterns).

If you go down this path you should check out FsUnit that provides some extensions to make testing in F# more enjoyable.

XUnit will feel very familiar due to the popularity of *Unit style libraries.

Composable testing

Fuchu
Expecto forked from Fuchu and re-architected. This style of testing framework is based on lists of functions/methods instead of attributes. This makes it easier to compose your test code.

I’m somewhat ambivalent to XUnit vs composable testing.

Going further down the test track

Property based testing

In property based testing you define the conditions for test data and let generators create that data to check your code against. These style of testing frameworks are excellent to combine with your existing testing.

When it comes to then add property based testing I’ve mostly used FsCheck.

Going deeper down the rabbit hole!

Say that you have gotten your feet wet and tasted some of the functional goodness of F#. Going further down you want to start to use some library to remove some of the copy paste. There are a couple of different libraries that can help you reduce your own infrastructure code.

F#X Extras

Contains a lot of helpful various utilities. Note monoid as an abstract class. It’s under the stewardship of fsprojects so accepts pull requests even though the original authors are not as involved.

ExtCore

ExtCore contains among other things combinations of computation expressions.

YoLo

YoLo isn’t really a library, it’s really a small file that gives you a bit of extra useful methods. Used by Suave to provide some common utilities for base components of F# like Option, Result and Choice.

F#+

My personal favorite. It’s a rejection of the assumption that F# cannot express higher level of abstractions. By using for instance monad transformers you can get the same functionality (combinations of computation expressions) as in ExtCore but without having to manually code it.

Conclusion

F# has a rich eco system of tools and frameworks. It can be a useful tool in order to deal with complexity or just to write something quick using some F# scripts. There are many paths that you can take to get where you need to go.

Is there a value in having diversity when it comes to solutions?

webmaster@gewalli.se (Oskar Gewalli) — Fri, 26 Oct 2018 11:01:51 +0000

Monoculture in agriculture has some definite downsides (lack of resilience, soil depletion, over reliance on fertilizer)
Monoculture in buildings can cause people to feel lost. Once you move in, you want to change your house to fit your family and your interests.

A suburb that started out with lots of very similar houses can end up with divergent colors, additions suited for different people, gardens with personal style. This should not be seen as a negative thing, it shows the history of the area and give a sense of identity to the neighborhood.

In programming we often have a drive towards standardization.

perhaps in order to increase the industrial or engineering nature of it?
To make it easy to get people from other teams to understand the code due to similarities in style

Having different styles/architectures/programming language

Since different software components suite different purposes, they should have style suited for different application / purpose. When you use components from different companies and open source initiatives, they will not fit your style but they will help you deliver.

One language to rule them all

There is talk about having a single language for frontend and backend. This type of talk has been going on for as long as I’ve been a developer. I’ve worked in such solutions: C#, Java or C++ as the single language. I’ve seen JavaScript on both client and server. This can be very helpful for people starting out. They use the same language that they are very familiar with in both settings. For one team this can be a good thing and help them deliver.

However compiling Java or C# to JavaScript have had some interesting challenges. The debugging experience, error handling, diagnosing errors. Source maps and other tools have helped make this less of an issue. Though since most frontend web libraries are written in JavaScript, it is often better to be familiar with JavaScript.

Writing JavaScript on the backend has had some challenges. The developers I’ve worked with have had it easier to work with C# (when it comes to deliver stable features). For simple services, JavaScript can be a good choice. There are also more mature libraries for Java and C# that fits more of server side development (for historical reasons).

Limiting yourself to one language also means limiting your recruitment base. Once people have started working, they usually pick up the other commonly used languages (in order to be able to modify other parts).

Hiring different style of programmers

Developers will find their own style of coding as they mature. Different people will work in different ways. Some write text then code, some draw diagrams then code, some tests then code, et.c.. When you hire people, most probably they will not be familiar with your domain and your standards. Just because they are not familiar with your preferred style, they can still prove useful (to your organization).

Consider frontend vs backend

When I started out, I didn’t hear about this distinction, while today it’s a big division. JavaScript was a loathed niche language. I think the primary reasons was that it was very difficulty to write working code. One big reason was due to browser differences between versions and vendors. Another reason was that linters was not in use. There are some gotchas that can be difficult to track down (unless you use linter). Then came jQuery (to smooth out the differences). Then came browser standardization.

Note however, if you write mobile apps or desktop applications, you can do that fully in C#.

Hiring people

How do you hire ok frontend web people? You go with JavaScript (since that’s the mainstream web frontend language right now).

How do you hire ok backend people? You go with Java, Python, C# or even PHP (since they are the mainstream backend languages right now).

How do you find great developers? One way is to go a bit niche. Interested developers will want to learn new things. If you have ‘cool’ tech you can attract interested developers.

Niche language

possible to hire great developers who have sought out the language in order to level up
harder to convince other people that they can work in such a language

very common language

easier to recruit people directly from school (since it’s common, schools target the language)
easier to recruit people from other parts of the industry
normal people, going to work and then going home, also called 9-5:ers or dark matter developers

Where should you have homogeneity

You need people to make sure that components (libraries/services) have internally uniform style. Having a good structure will make it easier to read the code. What kind of structure? I would say that it depends on the project and people working on it.

Why I prefer React over Angular

webmaster@gewalli.se (Oskar Gewalli) — Sun, 09 Sep 2018 08:18:42 +0000

React has a smaller scope than Angular.
I’ve found that my coworkers have an easier time grokking React than Angular: React contains fewer concepts than Angular.
Angular can sometimes make implementation of custom behaviors require a lot of deep knowledge (of Angular). For instance when implementing a custom dropdown or a checkbox.
If you are cheeky, you could say:

Angular is a somewhat over engineered platform, while React is sometimes used to create over engineered apps.

My personal opinion is that as long as you can use the existing behavior of the standard components (buttons, checkboxes, dropdowns et.c.), Angular can be a good choice. I’ve yet to have been in a situation where management will settle on not implementing something just because it will create additional complexity: Why my feeling is that if you are given the choice, you should go for React.

I’ve used knockout for several years as well (and still like it as a library). My biggest issue is that I’ve found that the documentation (and the API) leads people into using computed properties when they don’t need to. I would say that knockout is more well thought out than Angular (due to the smaller scope of the project). Due to the popularity of React compared to Knockout, I would guess that React would be a more natural choice.

F# 2 years later

webmaster@gewalli.se (Oskar Gewalli) — Sat, 08 Sep 2018 19:30:26 +0000

What’s the lure of F#?

The main lure of F# is that it’s a surprisingly competent ML style language that transcends some of the limitations of CIL.

From my point of view, the main distinguishing feature for F# is Statically Resolved Type Parameters. It’s a feature you don’t want to see in business code (same as reflection in C#), but something that enables libraries to take F# to the next level. Compare this to type classes in Haskell and what is called functors in OCaml.

What has delighted me?

The editor tooling for F# has grown more stable.
That F# has another perspective (compared to C#).
Type providers can simplify one-off “scripting”.

What has been annoying?

Personally most of the pain points have been in dealing with .net core.

Type providers have recently (2018/05/08) been fixed to work in .net core.
Using libraries intended to be consumed in C# that uses Microsoft.Extensions.DependencyInjection and Microsoft.Extensions.Options requires ceremony to use.

These are minor things. For the most part I don’t need dependency injection heavy code in the F# I’ve been writing.

What about Identity Server?

webmaster@gewalli.se (Oskar Gewalli) — Thu, 06 Sep 2018 05:52:29 +0000

My experience using IdentityServer4

See Identity Server.

I’ve (and others in my team) have worked on a project using Identity Server 4 since early 2017.

What lead us there?

The main reason why we started to consider it was due to the main business goals:

To have a JWT provider that enables a single page application to communicate with a plethora of micro services
To use an existing solutions. The reasoning being that if the company used existing software, that would create a dependency on that software instead of senior developers.
To have a single provider of identity and accounts.
To have the provider of identity and accounts as a micro service.

We noticed that asp.net identity together with Identity Server satisfies a lot of the specific business requirements.

Initial experience

We got something working quite fast that satisfied the requirements. We had an login, authentication and OAuth solution within days.

What happened next

Our first stumbling block was that there was a need for a JSON API to use with a back-office GUI that needed to be able to display accounts. Part of the trouble was that we needed mixed authentication: JWT and cookies. That turned out to be somewhat difficult with asp.net core mvc 1, but we eventually solved it. In asp.net core mvc 2 this has become much easier to configure.

Our next stumbling block was that there was a requirement that we could have a sign in flow from the single page application without disrupting the shopping experience. That requirement turned out to be more difficult to satisfy than the other requirements. In order to satisfy that we needed to dive deep into Identity Server in order to bend the opinions of Identity Server to the requirements.

Conclusions

The assumption that we didn’t need senior developers was faulty. Once we deviated from the assumptions made by Identity Server, we had to involve senior developers in order to implement the business requirements using a mix of OAUTH and other protocols.

Working on dotnet core on Mac OS X part 2

webmaster@gewalli.se (Oskar Gewalli) — Wed, 05 Sep 2018 18:48:47 +0000

After writing the previous post about getting some of the infrastructure working for non Windows, I’ve seen better ways of getting it to work. In particular we have seen the use of netfx.props as seen in FSharp.TypeProviders.SDK. Then you add the following to your csproj or fsproj file:

<Project Sdk="Microsoft.NET.Sdk">
  <Import Project="..\netfx.props" />

as can be seen in FSharp.TypeProviders.SDK.fsproj.

What makes for happy C#?

webmaster@gewalli.se (Oskar Gewalli) — Fri, 16 Mar 2018 10:51:45 +0000

One of the questions I’ve started asking myself is what makes for happy C#? After a few years getting a bit more understanding of ML like languages like F# I’ve started to realize that C# is not built to write the kind of code that you write in an ML. Instead, you are happier when you embrace the Ruby-like patterns in C#. If you look at C# sort of like a typed Ruby like language: Object orientation, mutability first and meta programming using reflection. When you see that this is shared between the two languages then more things make sense. I’ve noticed that good Ruby code translates very easily to good C# (although not always). Same is not true for good F# or Haskell.

Good Ruby code tries to keep a nice clean programming surface for the user (of a library).

Ruby is very object oriented. C# shares this trait since it is very adamant in that you need to have a class in order to have any code. Extension methods can be used to make a sort of monkey patching in C#. Monkey patching in Ruby can be dangerous, but applied right can help clean up your code. Same is true for extension methods. I would call a nice extension method to be polite: It only uses the extended type and any parameters sent in.

Architecture that fits C# and Ruby

Having smaller libraries with their own solution means that it’s easier for people to reason about the code. There are some downsides of this approach though: You need to keep libraries in lockstep when introducing breaking changes during a larger refactoring. You need to be careful about what kind of public API surface you accidentally introduce. It’s easy to introduce leaky concepts (concepts and abstracts that require you to write code to deal with a specific concept in many different places).

Writing micro services is another way of splitting up your code base. However, you need to know when it’s worth putting something in a separate library and when it’s useful to have it in a service. A service can easily introduce latency and give you bad performance. If you try to follow Martin Fowlers advice and apply that to your microservices, you can have relatively separate services (that they don’t cause performance issues because of hard coupling requiring many buss events or http calls to complete actions).

Having relatively small services (but not to small), means that it can be easier to reason about mutable first procedural code. I find it very difficult to try to reason about large a large code base with mutable instances.

Deviations

Since we have static analysis available through the compiler, we can use that to define value types and wrap strings, integers, et.c. when dealing with many different small id-types or values that are easy to mix up. I.e. value types of a single value. In Haskell you call this a Newtype. In C# you need to do a bit more work. If you have many of these types, you might want to create a t4-template for it. In Ruby you would (due to the dynamic nature of the language) not need a template for it.

Working on dotnet core on Mac OS X

webmaster@gewalli.se (Oskar Gewalli) — Sat, 17 Feb 2018 12:13:55 +0000

One thing I neglegted to mention was that there isn’t really out of the box editor support for targeting both net45 and netstandard1.6 (or higher versions).

My standard workaround for this have been to only target netstandard1.6 or lower for most of the relevant assemblies that I work on a day to day basis on. This works for most of the internal libraries since we deploy either netcoreapp1.1 or netcoreapp2.0 to Azure.

For libraries that are supposed to be consumed by a wider set that has been a bit of a pain to edit on Mac OS X.

If you try to open a .net core project that uses paket, the workaround to edit projects in Visual Studio for Mac is to disable automatic package restore.

The final workaround to be able to work net45 and netstandard1.6 dotnet core project on Mac OS X is to apply @enricosada’s fix to .bash_profile:

export FrameworkPathOverride=$(dirname $(which mono))/../lib/mono/4.5/

This has made it possible to open for instance Giraffe without doing any other workaround to have it working.

Dotnet core a year later

webmaster@gewalli.se (Oskar Gewalli) — Fri, 19 Jan 2018 20:21:06 +0000

We asked the other developers in December 2016 what they wanted to work on. They said that they wanted to work on dotnet core and MVC. I was a bit hesitant at first since that would imply using something relatively new to base the business on. A coworker at the time did inject that he was using it in production already.

It has been an interesting journey. From json + xproj to a new csproj format. The nice thing about the new format is that it includes packages in the csproj. The file format has been cleaned up (so that we no longer need to use albacore scripts to detect csproj issues. Using NuGet is less painful compared to previously (I feel less of a need for paket but NuGet still have catching up to do). Though, there is still a bit of weirdness that make it difficult for people. I’m helping a dev get into asp.net Mvc 2. Due to lots of different blogs about asp.net Mvc, asp.net Mvc Core 1, asp.net Mvc Core 2. This is similar to what we see on for instance differences between different minor versions of Angular. As the documentation for Mvc on microsoft sites is quite good, it becomes less of an issue than for NG.

We have used EF Core 1-2. I’ve previously used NHibernate. EF does not perform as well as NHibernate but is easier to start out with. I wish we had used Dapper since it’s easier to get performant and having to write SQL discourages the use of more complicated schemas. EF migrations doesn’t feel as elegant as some of the other migration frameworks, it does however work well enough due to the fact that it can be used on the command line.

What has worked well has been to use Swashbuckle.AspNetCore. It helps with keeping documentation in sync and cuts down on the need for hand written API cruft.

A lot of the API’s have been improved in asp.net core. Default dependency injection and logging abstraction, et.c. has made the C# code more pleasant to work with. That the configuration is using dependency injection instead of global values makes it easier to reason about configuration.

Running dotnet core 1.1 on appveyor and travis

webmaster@gewalli.se (Oskar Gewalli) — Sun, 29 Jan 2017 08:05:29 +0000

I wanted to see the state of EntityFramework on core. Turns out that it works on other platforms than Windows! NHibernate worked on mono as well as Windows, and now Entity Framework has caught up with that. However, EF Core is still somewhat limited as a ORM. This should not matter much if you are writing relatively simple SQL (LINQ) and have few relationships (so that you have less of a need for more advanced constructs). Many business apps do fall into this category if the requirements are made simple and clear (one source of complexity is fuzzy requirements leading to confused code).

appveyor

Running .net core on appveyor turned out to be quite easy. They have already installed .net core 1.1 on their servers. All you need to do is to run the dotnet commands.

.net core official .travis documentation

The first thing I tried was to try out a configuration based on the travis documentation about running .net core on travis. This should work for a 1.0.0 version, however, I had dependencies on core 1.1.0. Note that this type of configuration is also used by Enrico Sada in the dotnet docs:

$ dotnet test
Project Tests (.NETCoreApp,Version=v1.1) was previously compiled. Skipping compilation.
The specified framework 'Microsoft.NETCore.App', version '1.1.0' was not found.
  - Check application dependencies and target a framework version installed at:
      /usr/share/dotnet/shared/Microsoft.NETCore.App
  - The following versions are installed:
      1.0.0
  - Alternatively, install the framework version '1.1.0'.

Trying other versions of dotnet core on travis

I tried with a later travis dotnet version:

dotnet: 1.0.0-preview2.1-003155

This resulted in:

$ dotnet test
Project Tests (.NETCoreApp,Version=v1.0) was previously compiled. Skipping compilation.
The specified framework 'Microsoft.NETCore.App', version '1.1.0' was not found.
  - Check application dependencies and target a framework version installed at:
      /usr/share/dotnet/shared/Microsoft.NETCore.App
  - The following versions are installed:
      1.1.0-preview1-001100-00
  - Alternatively, install the framework version '1.1.0'.

Trying the scripts from EF Core

I tried to run a setup based on the .travis.yml in the EF repository. The scripts failed while running locally, so I abandonded that attempt early (perhaps they need environment in the repository that I’m missing).

Trying out scripts from blog

I tried running scripts found on Andrew Locks blog. These failed with:

/home/travis/build/wallymathieu/entity-framework-studies/.dotnetcli/sdk/1.0.0-rc4-004706/NuGet.targets(210,5): warning MSB3202: The project file "/home/travis/build/wallymathieu/entity-framework-studies/Web/Web.xsproj" was not found. [/home/travis/build/wallymathieu/entity-framework-studies/entity-framework-studies.sln]
/home/travis/build/wallymathieu/entity-framework-studies/.dotnetcli/sdk/1.0.0-rc4-004706/NuGet.targets(210,5): warning MSB3202: The project file "/home/travis/build/wallymathieu/entity-framework-studies/Tests/Tests.xsproj" was not found. [/home/travis/build/wallymathieu/entity-framework-studies/entity-framework-studies.sln]
/home/travis/build/wallymathieu/entity-framework-studies/.dotnetcli/sdk/1.0.0-rc4-004706/NuGet.targets(97,5): warning : Unable to find a project to restore! [/home/travis/build/wallymathieu/entity-framework-studies/entity-framework-studies.sln]

and

/home/travis/build/wallymathieu/entity-framework-studies/Tests/Tests.xproj(7,3): error MSB4019: The imported project "/home/travis/build/wallymathieu/entity-framework-studies/.dotnetcli/sdk/1.0.0-rc4-004706/Microsoft/VisualStudio/v14.0/DotNet/Microsoft.DotNet.Props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

Probably due to being for a previous version of dotnet core.

Trying out install instructions

Instead of trying out travis specific instructions I tried to copy the instructions found for installing net core on ubuntu. Since these instructions are probably maintained, they worked without a glitch. I ended up with this version of .travis.yml:

language: csharp
sudo: required
dist: trusty
addons:
  apt:
    packages:
    - gettext
    - libcurl4-openssl-dev
    - libicu-dev
    - libssl-dev
    - libunwind8
    - zlib1g
env:
  global:
    - DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
    - DOTNET_CLI_TELEMETRY_OPTOUT: 1
mono:
  - latest
os:
  - linux
before_install:
  - sudo sh -c 'echo "deb [arch=amd64] https://apt-mo.trafficmanager.net/repos/dotnet-release/ trusty main" > /etc/apt/sources.list.d/dotnetdev.list'
  - sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 417A0893
  - sudo apt-get update
install:
  - sudo apt-get install dotnet-dev-1.0.0-preview2.1-003177
script:
  - dotnet restore
...

Update

Note that the version I referenced while trying out the install documentation is higher than the version I tried out. Trimming the yml resulted in the following travis yml:

language: csharp
sudo: false
dist: trusty
dotnet: 1.0.0-preview2.1-003177
mono:
  - latest
os:
  - linux
script: 
...

The usefulness of generalized regular expressions

webmaster@gewalli.se (Oskar Gewalli) — Wed, 12 Oct 2016 18:30:07 +0000

It’s not obvious at first thought that there are applications for generalized regular expression engines. During my career, the regular kind has been the most prevalent kind. For certain problems, there is a simple pattern matching, that regular expressions excel at. For instance, in the case when you have a matrix with relatively regular content, it can be thought of as a general type of expressions. I’ve created the library Zander to help with that type of problem.

Should you work with old technologies?

webmaster@gewalli.se (Oskar Gewalli) — Mon, 15 Aug 2016 05:49:37 +0000

Why should a coder avoid old technologies?

Large complicated systems are often written in older technologies

Newer technologies can mean assignments with fewer lines of code and more greenfield development. Since the amount of code is proportional to the amount of time and people spent working on that code. Greenfield development is easier than working with millions of lines of code. It can however be an interesting challenge to rewrite an older system.

Market value

In order to sell consulting services, many people will push for new tech.

Since the market value of a coder is related to experience with the current technologies in use, knowing and working with new technologies is a good policy to stay competitive. This means that even regular employees will push for new technologies.

New tech may shift how you develop. For instance JavaScript performance improvements have made it feasible to run more logic browser side, causing a drive towards writing code more like regular server side code using ko, angular, react et.c.. Virtual machines performance that is close to native performance (JVM, CLR) and the use of GC can simplify a lot of code (easier to work with) and create demand for different skills.

Writing new libraries

Working in mature technologies can mean that the trailblazers have solved many of the problems. What’s left is to try to get patches for potentially unmaintained libraries. If you are one of the trailblazers you can increase your market value by providing a library that many people use (thus people get to know your name). You will need to market the library in order to gain attract developers to write pull requests and contribute issues. Good way of marketing a library is to present it during meetups or conferences.

Why should a coder avoid untested technologies?

Stability

New code without tests or insufficient test coverage has a greater probability of bugs. As development of new things progress, the API’s will change until they mature. The maintainers may wish to iron out quirks from the API’s.

Learning overhead

If the new technology is radically different from the old (angular vs jQuery), the amount of time spent learning impacts your ability to deliver. This cost can be sufficiently small if you know that you will have a lot of things to deliver (i.e. you know that you will spend enough time on the code). In order to avoid information overload you might want to have a mixture of known and unknown. Some developers are better at grokking new libraries and languages than others, but you still need to think about the team.

Trailblazer moving on to greener pastures

A trailblazer might want to move on to the next language after creating a few new library. Or if the technology becomes somewhat popular (without people that want to help out) the maintainer might not have enough time on their hands to support the library. This means that some libraries might become abandoned. Talk with the original author, sometimes they can be convinced to pass on the keys to NuGet, github et.c. as long as you are willing to help maintain the lib and review pull requests.

What does technical debt mean for a project?

webmaster@gewalli.se (Oskar Gewalli) — Sat, 02 Jul 2016 06:20:29 +0000

Technical debt is a concept that suggests that the cost of developing software has been postponed. For instance it can be the conscious act of delivering features while taking shortcuts (in order to get them delivered at a certain date).

When changing a component, the amount of bugs introduced is often related to technical debt of that components. Higher technical debt means greater likelihood of bugs.

The time to resolve various issues is related to the technical debt. Technical debt will express itself in the sense that some tasks that should be simple to complete take a lot of time to finish.

Estimation: There is a risk that after something looks implemented, there are Y number of bugs to be found. It may mean that after one task seems to be done, issues and bugs related to the component will crop up where each issue take on average X amount of time to resolve.

If nothing is done to pay off technical debt, you have a situation where after an issue A1 you have Y bugs. Each bug takes an average X amount of time to resolve. After each bugfix there is a risk of a new bug. Since the technical debt is not repaid, we see that the average time X will increase. It means that you have a similar situation as when a person with a loan allows the interest rate to accumulate (ie debt increases).

How do you achieve profitability in a project with high technical debt?

Lower customer requirements in existing parts of the system. If you lack people that can hold off customer demands for changes in existing parts (requirements that entail greater complexity), customer requirements will drive increased costs.

Build separate modules where technical debt can be kept low. However, this can mean that the greatest cost (time spent) will be related to integration code (code to connect between things) between to connect modules of low technical debt with modules of high technical debt. However as long as you avoid changing modules with high technical debt, you can avoid cost increases.

Hire people that take a long term view (see the section about how to handle technical debt) in order to pay of the loan over time. Together with the developers, make sure that they create a long term plan for how to reduce the debt.

What does technical debt mean for people working in the project?

The complexity and difficulties to deliver, can along with demands for quick deliveries create severe stress: Morality is reduced. This means that the probability of people leaving the project increases. The likelihood that people within the project burnout increases. What does this cost? According to DN, it can end up at about 400 000 SEK (ca 40 000 EUR).

How do you handle technical debt?

Doing code reviews with experienced developers
Automatic testing to reduce the likelihood of regression
Requirements management: ensure that the requirements reduce complexity (eg by having a dialogue to arrive at simpler solutions)
By identifying the problem through different tools and working to isolate or remove the parts of the code that need to be changed due to customer requirements
Refactor parts that are covered by tests
Create new modules that strangle old parts of the system

How do you identify the problem?

Static code analysis can help identify problem areas. For example .net are:

NDepend
sonarqube is able to analyse C# code (with some effort).
Resharper
Selected rules in StyleCop

Test coverage percentage can give you a picture of the areas where changes have higher probability of regression (ie can result in extra work).

Common enterprise architectures

webmaster@gewalli.se (Oskar Gewalli) — Fri, 03 Jun 2016 18:24:52 +0000

Some patterns I’ve usually encountered while working code bases written by .net developers. You can also read more on msdn.

Domain driven design inspired

Usually you find classes with postfix name like:

Repository : List like interface for working with persisted data. Usually state of fetched objects is tracked, so you can do customerContext.Save() to persist changes to objects.
Service

People are often confused by “Unit of work” also mentioned in connection to these patterns. Many OR-mappers implement the repository pattern. Popular libraries include Entity Framework, NHibernate and RavenDB (library and database).

Key concept

The key point of DDD is ubiquitous Language. Being able to have roughly the same type of concepts as the users can be very helpful.

DAL

This type of architecture is quite simple. The main benefit of it is that it’s easy for people to grok. This type of architecture is not as popular anymore because it increases the amount of code you have to write.

Manager : An object that does things by using other managers and accesses the data using DAL
DAL : Data access layer. State for fetched objects is not tracked. You have to make the DAL aware of changes customerDAL.Update(customer);
DTO : Data transfer object.

In this type of architecture it’s not uncommon to have an additional layer in the database implemented by stored procedures.

Message queue & Service Bus

Popular choices I’ve encountered are RabbitMQ and MSMQ. A popular library is MassTransit.

Provides a way to lower cohesion between components. Can also be helpful if you want to distribute work.

Merging in svn

webmaster@gewalli.se (Oskar Gewalli) — Sun, 29 May 2016 11:46:21 +0000

While working as a consultant at a company that uses svn, I needed to keep a branch in sync with the master branch. This turns out to be a lot of work.

Problem was that it was not possible to just clone the repository using git svn. The clone failed after a while (I think due to the long connection time). First you might want to read up on it from this blog post.

The command I used was something like:

git svn clone -r 10327 --username=oskar.gewalli http://company.com/scm/svn/Project project  --stdlayout --prefix=svn/

In order to fetch changes I did:

git svn rebase

And to push changes:

git svn dcommit

Since I could not clone the repository and get the full branches, I had to add branches manually. It took little bit of work but was described on stackoverflow.

The value of this work is that merging branches is far easier in git compared to svn.

Going deeper into angular

webmaster@gewalli.se (Oskar Gewalli) — Tue, 12 Apr 2016 17:32:58 +0000

After getting to know details about ng for the last months, I’ve come to appreciate some of the features of angular. You can write quite clean looking code if you know enough ng. From my personal point of preference, “scope” is a bit all over the place. I remember using angular for small project in 2012 not thinking much of it (angularjs shines for projects with a limited complexity).

Before trying to write an angular app

In order to get sort of a grasp of it there are things that you should read like ng patterns. Some of these things won’t make sense until you understood some ng weirdness: nested scopes in angularjs You may also need read up on the opposition: angularjs the bad parts.

Understanding angularjs

Once you have read these things, you should be able to make some sense of the ng1 and ng2 code bases. There is quite a lot of documentation mixed in with the source (used to generate the api docs). It’s probably better to start to read the code from the point of the api docs (if you know what area you’re interested in), since some of the code has more documentation than code. If you are planning to write something bigger (>10k lines) in TypeScript, it can be worth to take a look at the ng2 code base since it’s an open source example of a large TypeScript code base and scripting language usage of files.

The future

There is a lot of focus on components in angular2. In earlier versions (prior to ng 1.5) you can achieve similar type of code by creating directives composed of template and controller.

My guess is that angularjs will have a sort of stable position (but it’s definitely not hip anymore).

Clean your work area!

webmaster@gewalli.se (Oskar Gewalli) — Sun, 24 Jan 2016 06:34:44 +0000

When in the kitchen, how are you behaving to old dishes? Do you make space in order to have space to cook the food? After finished cooking the food, do you leave dirty dishes to the next day?

I’ve worked in a couple school kitchen. When I worked, it would be in the form:

make sure that there are nothing cluttering the kitchen that shouldn’t be there
make sure that the kitchen is clean
prepare food and put food and milk in place for the children
do dishes
clean up the kitchen

I’ve worked a few months in a mechanical workshop. My boss told me how to work:

make sure that the area where you are going to work is uncluttered and clean
work
make sure to put back all the tools
put any unused materials back to storage
clean up the work area

My grandfather (engineer and architect) told me how to behave in his workshop:

clean up the table, make sure you have space to work
work
clean up the table

In his mind, if you go for lunch, then work is considered finished (so you need to clean up before going to lunch).

In software development it’s not unusual to try to ignore the cleaning part of the job. Why is this considered OK? Will the job be easier if you have to slog through:

commented out code
comments that are unmaintained
debug code
dead code
uncommunicative function names

and the list goes on.

Why do you avoid to follow even the boy scout rule of leaving it a little better than you found it? Perhaps it’s because you have no sense of code smell? Perhaps it’s because you’re stressed? You’re a junior developer?

The thing about code is that once you have a clean working area (clean part of the code base where you are working), is that changes and additions will become easier to implement. Fewer things to keep in mind will keep your working memory focused.

Divide and conquer

webmaster@gewalli.se (Oskar Gewalli) — Sat, 16 Jan 2016 12:06:00 +0000

This approach to programming is not only useful for algorithms, it’s also a recurring architectural theme. The current version is called microservices. In unix you have the small command line tools that does one thing well. On a smaller scale it’s about splitting your code into several files and then folders. When you have identified a need for a library you might create a nuget, gem, maven or npm package (see modulecount).

Inheritance in JavaScript

webmaster@gewalli.se (Oskar Gewalli) — Fri, 11 Dec 2015 16:06:44 +0000

How do you inherit from a “class” in JavaScript

Especially if you have a class that is a mix of constructor function and prototype methods and properties.

"use strict";
function getSeq(start){
 return function next(){
   return start++;
 };
}
var getId = getSeq(1);

function Mammal(name){
    var self = this;
    var id = getId(); // private variable
    this.getId = function(){
        return id;
    };
    Object.defineProperty(this, 'name', { // we want name to be mutable
        get:function (){ return name; },
        set:function (value) { name = value; }
    });
    this._old_name = name; // this value will be immutable

    // we want for the object to implement the same interface
    // this is a way to make objects more like java or C# objects
    Object.freeze(this);
}

Mammal.prototype.says = function(){ // but we can still add functions by adding to the prototype
    return this.saying || '';
};

Some notes about the above code

We (me and Peter Wilhelmsson) have found that the usage of “use strict” together with Object.freeze and Object.defineProperty can some JavaScript confusion (accidentally setting a new variable when you want to set an existing, hard to find initialization of objects). Usually as with many old languages with many features (c++, perl) you should choose a subset of that language to simplify for other developers.

By modifying the prototype you can always add methods even if the object itself is frozen.

How do you create an instance of this “class”?

var m = new Mammal('Missan');

How do we inherit from this “class”?

"use strict";
function Feline(name){
    var self = this;
    this.saying = ''
    // the last thing we do is call the base class in order to inherit and freeze this class
    Mammal.call(this,name);
}

Feline.prototype = Object.create(Mammal.prototype); // we want the methods and properties added through prototype
Feline.prototype.constructor = Feline; // but still have the correct constructor

function Cat(name){
    var self = this;
    Feline.apply(this,arguments);
}

Cat.prototype = Object.create(Feline.prototype);
Cat.prototype.constructor = Cat;

Naming of JavaScript classes

Note that when you write:

namespace.Name = function (){ }

This becomes an anonymous function (in many browsers). In order to give it a name we need to write it like:

namespace.Name = function namespace_Name(){ }

This can be done as a post processing step in your grunt or gulp with logic similar to the following:

"namespace.FunctionName = function() {};".replace(
 /namespace.([^ ]+) *\= *function *\(/, function (str, gr){
   return "namespace."+gr+" = fucntion namespace_"+gr+"(";
 });

Using pure prototype inheritance

If you are using a library that does the right thing when it comes to “this”, then you can add methods to the prototype instead of “this” as a captured variable named “self”.

webforms and confusion

webmaster@gewalli.se (Oskar Gewalli) — Fri, 30 Oct 2015 11:41:16 +0000

webforms and 3000 lines of code-behind

I’ve noticed that the common failures while using asp.net webforms is related to people getting confused about the asp.net page and control life cycle. Since there are so many different events to keep in mind, it’s easy to place code in the wrong place and get something that sort of works (but does not work in all cases). If you are stressed, you might add a small if statement to fix this problem in the special case, increasing complexity in the code behind file. This then turns into 1000-3000 lines of hard to read C#. I’ve found that building small user controls that are built according to how you should build user controls, can help you avoid this mess (the technical lead at Macom when I worked there pushed for this). This requires you to learn how to write user controls though.

I get the feeling that it’s not unusual for developers have a hard time understanding how to interact with different components written in different files. Maybe this is why junior developers using this type of code can create a hard to change architecture: large files with a lot of interconnected logic inside each file.

mvc and JavaScript

Writing asp.net mvc is conceptually easier (there are fewer things to learn, since the focus is different). However, if you want the same functionality you might have to add some JavaScript in order to get an app (as you get in asp.net). I have found that most developers get confused when reading and writing JavaScript (unless they have enough experience using it). This confusion can be mitigated by using a subset of JavaScript or a transpiler (problem is that you need someone in the team that knows JavaScript in order to be able to make informed decisions about it). In some cases you might write more JavaScript in order to reduce the overall complexity.

JavaScript maturity

I find that the difference between C# and java is relatively minor right now. The main feature I missed in java has been lambdas (or lambda like syntax). Ruby, python and php all are similar in the sense that they have sort a class concept and that you can inherit from other classes. There is something like “this”, so that you can access the object. In JavaScript however, “this” does not mean the current object (it can be, but it’s not always the case). However, since most server-side developers are forced to learn JavaScript in order to complete most assignments, I would imagine that many developers know JavaScript and one of the other languages that are more similar.

We see many concepts that have been promoted server-side entering client-side applications as well. When working with a larger JavaScript codebase then the problems that you face will be familiar to working with a large server-side codebase.

conclusion

What could the conclusion be from this? If we see es6 with classes and modules and the adoption of frameworks that work with objects that use this their functions to access the object itself, then programming JavaScript compared to server side will feel less different. Some of the JavaScript frameworks out there require you to learn many things (and are then compared to webforms). My feeling is that it’s probably better to avoid larger frameworks like angular and extjs (unless you and your team is prepared to learn a lot of things). It’s better to use well known small and stable libraries (I’ve heard the same opinion being voiced by others so this is nothing new).

ruby on windows

webmaster@gewalli.se (Oskar Gewalli) — Fri, 16 Oct 2015 05:57:52 +0000

I’ve had some trouble installing gems on windows instances using bundle install. I had this problem first at ~2009. The problem is missing certificates.

While this fix is correct, a quick and dirty approach is to make sure that your gem files have source http instead of https:

# A sample Gemfile
source "http://rubygems.org"

gem 'albacore'
gem 'nuget_helper'

This means that the communication to download gems from rubygems is not private.

Mono and OS X El Capitan

webmaster@gewalli.se (Oskar Gewalli) — Fri, 16 Oct 2015 05:32:05 +0000

After installing OS X El Capitan on my computer I noticed that fsharp interactive failed to start because it failed to locate mono:

/Library/Frameworks/Mono.framework/Versions/4.0.4/bin/fsharpi: line 24: exec: mono: not found

Session termination detected. Press Enter to restart.

If we look at the file at that position we find that it assumes that mono is in the path. I checked /etc/paths, and it looked correct (i.e. based on the paths file, mono should have been in the path):

mathieu$ which mono
/usr/local/bin/mono
mathieu$ ls -la /usr/local/bin/mono
lrwxr-xr-x  1 mathieu  admin  48 Sep 12 21:56 /usr/local/bin/mono -> /Library/Frameworks/Mono.framework/Commands/mono

I’ve found that there is a stackoverflow answer. I opted for a somewhat different approach and created a file in my ~/bin folder with essentially the same fix (this was before the stackoverflow answer, so it’s not exactly alike). You can find it here.

After that I changed the xamarin studio F# interactive settings so that the path is ~~~/Users/mathieu/bin/fsharpi~~~.

I’m thinking that later on I want to change the settings for fsharpi and fsharp to point to a local nuget repository (containg a paket.dependencies file) with FSharp.Compiler.Tools installed.

Using owin to implement an api

webmaster@gewalli.se (Oskar Gewalli) — Mon, 31 Aug 2015 15:52:32 +0000

I tried implementing the same customer service xml api that I’ve implemented in sinatra, rails, C# webhandler, nancy etc. I was pleasantly surprised how easy it is to get it up and running. There is a story for how you write tests: TestServer, that you install by referencing Microsoft.Owin.Testing.

Previous times that I’ve tried using Owin on mono, there has been some problems getting it to run (I think it was something related to a missing dll). Now it works fine. :)

You can find the code here: CustomerService/Owin

path unshift dirname of file

webmaster@gewalli.se (Oskar Gewalli) — Mon, 31 Aug 2015 12:17:46 +0000

Instead of using:

$:.unshift File.dirname(__FILE__)

to add a folder to the path, a better way inside your ruby code is to use:

require_relative 'subfolder/file'

in order to require files. You can also use a relative path in require (i.e. ‘./subfolder/file’).

BDD style testing in qunit

webmaster@gewalli.se (Oskar Gewalli) — Mon, 31 Aug 2015 08:52:38 +0000

Before the vacation I was working with Karl Adriansson on a project as the front end programmer. I helped get a front end JavaScript parts back on track. While doing this, Karl and one of the testers supplied me with specifications from the requirements. Since the specifications where very detailed, It was very easy to transform them into sort of BDD style tests:

test("PROJ-783", function (){
  var ctx = given_a_customer_previously_bought(black_sweater());
  and_I_currently_have_X_of_Y_in_stock(ctx, 3, black_sweater());
  when_he_returns_product_X_for_refund(ctx, black_sweater());
  then_I_should_have_X_of_Y_products_in_stock(ctx, 4, black_sweater());
});

Note that I’ve taken the specification from wikipedia in order to have a meaningful sample. We used the convention of writing the jira id as the test name in order to know from where the specification came.

Since we are using ordinary language constructs (methods that take parameters), the level of abstraction here is relatively low. Many of the helper methods can in many cases be reused since you might have many specifications related to the same subdomain. Some of the helper methods will only be used once. In some cases you might want to drop some of the prefixes in order to avoid to much duplication.

Build tools and package managers for .net

webmaster@gewalli.se (Oskar Gewalli) — Thu, 27 Aug 2015 10:49:54 +0000

What criteria do I use

I prefer for the build tool to work on both *nix and windows. I do some of the .net coding on os x. The build systems mentioned here are all related to working with .net (and all except psake work in *nix).

If you are coding .net

What package manager to use?

Right now there is NuGet. There is no good reason why you should deviate from using that format. However, you can choose to use NuGet.exe, NuGet in visual studio with packages.config files or use paket with ‘paket’ files.

Paket assumes that you want to have the same version of a package in all your projects in a solution. There is also a bootstrapper so that you don’t need to install NuGet.exe on you build environment or add it to the repository (the paket.bootstrapper is only 27.5KB). Paket and FAKE makes a nice package (an example can be found here).

What build tool to use?

I would generally unless there is some good reason avoid writing msbuild files. They are better suited to describe projects (csproj).

Gulp it’s Grunt

Since there is a lot of tools available for web developers that leverage grunt, many .net developers use grunt. It works, the api is a bit odd (json can be used to configure tasks). If you code js for other reasons, then the callbacks will probably not bother you. Note that there is another js build tool called gulp. Gulp is smaller and more focused on streaming builds (if you have looked at the ideas from reactive extensions you will find it familiar).

I’ve tried to use grunt. Node is quite easy to install on windows.

Rake

I prefer to use ruby’s make like tool rake for building open source C# apps since I usually have ruby installed wherever I work. Rake has some nice syntax that looks quite clean. It requires you to know some ruby though.

Usually when using ruby you will want to use bundler to manage dependencies.

Faking it?

Fake is a build tool where you write F# scripts. For F#-project this feels like a nice way to go. Fake is delivered by NuGet (so you won’t need anything extra installed). It requires you to know some F#.

It works well under mono. I’ve not had any troubles with it.

The syntax is a bit unfamiliar, but that can be the case with F# written APIs (using operators like ==> or »=).

PSake

Writing build scripts in powershell. This makes a lot of sense. Many times when you write build scripts, you are scripting windows servers as well; so you will have some experience using powershell. However, powershell is a bit of an odd language compared to many other common languages.

If you are doing windows only work, then this tool is valid.

The Cake is alive?

There are many different C# build tools. The most popular right now looks to be Cake. You don’t have to install cscript in order to Cake or Nake, but can download a .net exe from NuGet. You can use this exe to execute build script files. Sake is installed via chocolatey. Cake-files are actually a different language: a superset of C#. This means that visual studio and monodevelop etc wont work as expected. It will let you write build scripts in a C#-y syntax.

FSharpx.Extras

webmaster@gewalli.se (Oskar Gewalli) — Sun, 23 Aug 2015 18:44:46 +0000

Turns out that there is a simpler way of dealing with optional values in C#.

Include FSharpx.Extras to your C# project and then you can do the following:

optionalValue.Match(
    ifSome: value => ProcessValue(value),
    ifNone: () => NoValueToProcess()
);

Thanks to extension methods in this library. There are extension methods for other common discriminated unions as well.

F# 2 weeks later

webmaster@gewalli.se (Oskar Gewalli) — Fri, 21 Aug 2015 15:17:39 +0000

Since I’ve been without an assignment I’ve had the time to look into F# more.

Where did I find material?

Why did I start looking into F# again?

I share the office with Fredrik Forssen. I find his love of F# inspiring.

Things I’ve found that delighted me:

Less ceremony to declare classes, records, methods and functions (when you get used to how you write it)
Easier to compose things (for instance if you want to create a chain in order to use many different classes together for an import)
Monad support (and yes, I know that these things can very easily be abused)
Package manager and Build tool that works well together (paket and fake).
Records defined in F# are like hand coded readonly classes with IEquatable and IComparable implemented

I’ve previously found the following sweet when looking at the language:

Immutable by default
Language construct to create a clone of a record but with different values for some fields
Language construct to match more than hard coded strings, enums and integers

If you forget to add an equals sign to a type or module declaration you get weird compiler errors
Gui tooling is still a bit alpha (the command line version works great)
It’s easy to create a F# project where you need to manually edit the fsproj file in order to get it to work on mono and windows (this was a problem in the early days of NuGet)
Almost all the library methods in .net becomes a tuple parameter in F#, this is a bit confusing since it looks like an ordinary function call in C#
Exposing F# Option on your types makes it harder to interact with your F# code from C#

    using OptionalUrl = Microsoft.FSharp.Core.FSharpOption<Customers.Url>;
    ...
       var customer = new Customer(
          ...
          pictureUri: OptionalUrl.Some(new Url("http://someurl.com/something?some=asd"))
       );

Working with spreadsheets

webmaster@gewalli.se (Oskar Gewalli) — Sun, 07 Jun 2015 20:42:15 +0000

Excel html table hack

I’ve used the classic web hack where you send an html file with excel mime type (and file ending) containing a table. I’ve had trouble with excel being more particular about allowing this hack nowadays. Another problematic aspect is if a user sends you such a file without re-saving it. Since we have open xml based formats for spreadsheets, there is no need for this hack.

CSV

I’ve used different csv-like formats. These have the con of being dependant on the language settings (one partial fix is to detect the user language settings and render different formats). If you are using more than ascii you might also need to do a binary comparison of an excel generated csv and what you generate. Since csv looks fairly simple, many write a short regex to parse it. This has a tendency to devolve due to the different ways that you can implement csv. You can get into trouble when you are begin to support character escaping and newlines. I prefer if you can keep this kind of messy logic out of your otherwise relatively clean domain code (for instance by using a different language ). By using a different deliverable to transform data into predefined format you have an easier time to use many different libraries to parse weird formats.

Xml based (.xlsx)

Due to the popularity of JavaScript we have some libraries that you can use to generate xlsx files on the client-side. I’ve tried excel-builder and have been able to use it. From a programmer friend I’ve heard that he has used exceljs in order to generate excel files on nodejs. The documentation of excel-builder is lacking, but it was easier to get it to work than js-xlsx. In order to work with excel-builder I read the excel-builder source and compared excel-generated .xlsx files with what the program generated. Most of my problems was due to the client specifying how the excel file should be styled.

Lots of deliverables

webmaster@gewalli.se (Oskar Gewalli) — Sat, 06 Jun 2015 19:31:41 +0000

The use of many assemblies has an appeal from an architectural standpoint. In order to separate dependencies you want different parts of your code to be able to load even though a certain version of visual studio sdk is not installed. To have a fixed api implemented by adapter assemblies for many different versions of a hard to install library dependency is a way to allow work on code without having to install every single version of the dependency.

The problem is if you separate your code into many assemblies to early before you know the interdependencies between different subdomains. You can easily get a lot of code and extra assemblies due to subdomains that need to know much about each other. I like the reasoning in Martin Fowlers post: That you should wait until you know more.

Sometimes, the best thing is to start with a new library or dll. Starting out a bit greenfield when you start develop something new (something of sufficient size and with sufficiently small amount of knowledge about the existing code). It’s like applying the single responsibility principle on a larger scale.

Thoughts on using Angularjs

webmaster@gewalli.se (Oskar Gewalli) — Sat, 06 Jun 2015 17:06:11 +0000

I’ve done some minor work using angularjs. Angular looks like a promising direction for certain type of applications.

There is a lot of drama surrounding ng2. Some of is due to the fear that ng1 applications will become obsolete when ng2 comes around without a good migration path. Jon Skeet recently blogged: backwards compatibility is still hard. It will be interesting to see if the ng team manages to create a nice migration path from ng1 to ng2. Perhaps we will see ng1 having a continued existence due to the proliferation of applications written in ng1. The python community has struggled for years with python3 and the migration from earlier python versions. However, the complexity of most languages far exceeds the complexity of ng framework (but the differences between the versions looks more prominent).

You can be productive working with ng (as with many other frameworks). This type of framework will probably stick around for a few years until someone comes up with a better way of doing client applications in JavaScript. I’ve heard criticism from Eisenberg regarding ng2, but find that Aurelia looks very similar to ng.

What I’ve heard from people that have done more consulting on ng is that for complicated code the ng model of detecting changes can cause performance issues. For many business type of applications you don’t need that much calculations or complicated business logic. Some people have found ng to be hard to grok. I’ve talked to a former coworker who has started working with backbone. Backbone is easier to learn. You can read the entire annotated source of backbone. This is only natural since ng has a lot more code. You might notice how ng compares to backbone is similar to the difference between rails and sinatra. Rob Conery compares ng to rails.

So, should you use ng on your next project? As always, it depends. For some applications a rails type of frameworks on the server-side is the best fit (big opinionated frameworks that tell you how you should structure you code). For others, a more sinatra like approach (smaller framework focusing on fewer things). Being stuck in a predefined mould can be stifling if your domain does not lend itself to that shape. If it does, you can be fairly productive.

travis supports csharp

webmaster@gewalli.se (Oskar Gewalli) — Sun, 22 Mar 2015 19:16:19 +0000

I’ve used travis to test C# projects for a few years now. It has been a bit of a pain to get it working (read custom ppa or setting language to objective c and downloading and installing package intended for xamarin studio).

I feel that good things happen because people have more confidence in mono.

How do you test your projects on travis? Read the travis documentation or look at how I have done it in for example with. I’m using ruby to build C# since it’s cross-platform. You can substitute rake for grunt if you prefer nodejs.

Another way is to use that bash has some nice filename expansion. This means that you can probably write:

mono ./src/packages/NUnit.Runners.*/tools/nunit-console.exe

to avoid hard coding the version of the runner.

Kibana vs Elmah like gui

webmaster@gewalli.se (Oskar Gewalli) — Fri, 13 Mar 2015 22:05:13 +0000

The nice thing about using Kibana for your logs is that you get nice custom filtering. Thus it’s very easy for others to only view errors they feel that are relevant to them.

If you have a many systems (gui’s, services, etc) where you pipe a lot of log data into the same log database, then it can be beneficial to to filter out systems that you’re not coding anything in.

The downside of having easy filters is that it’s really easy to ignore errors. A screen with the graph of all errors could perhaps mitigate that.

It would be nice to have Elmah like formatting of stacktraces. Perhaps a chrome plugin could fix that. Perhaps port the stacktrace parsing from ElasticElmah to js.

The nice thing about having your logs in elastic is that you don’t need to just use kibana to query your logs. It’s really easy to use Nest to do queries. Unless you have huge amount of logs you can do groupings in memory.

Event sourcing and in memory database

webmaster@gewalli.se (Oskar Gewalli) — Tue, 03 Mar 2015 20:38:37 +0000

Trying out OrigoDb, I found it interesting to try out a more raw file store approach and NEvent Store. Why the connection Event Sourcing and In memory databse? Mostly because I tried out OrigoDb first. In memory database feels like a nice fit for Event Sourcing (since the log that can restore the database can be persisted in an append only manner).

Many of us have heard or read Martin Fowlers Event Sourcing. He mentions in memory databases in the article as well.

OrigoDb is a bit easier to setup compared to NEvent store. NEvent store is more mature. I like that you don’t need to inherit from a base class in NEvent Store. It does mostly what it says it does: Persists events. This means that it if you want to use it for an in memory situation you probably should make the objects readonly. The OrigoDb approach of using inheritance gives you fewer lines of code (since you don’t get command and handler types). NEvent store is more talkative and requires more code to handle events.

Since OrigoDb assumes that you want help with making absolutely sure that your in memory data is not corrupted by a assignment of on an object you need to interact with the it in a slightly different manner (from OrigoDb CustomerDataTests ):

        [Test]
        public void CanGetCustomerByFirstname()
        {
            var customers = _engine.Execute(m=>
                m.QueryOverCustomers()
                .Where(c => c.Firstname == "Steve")
                .ToList());
            Assert.AreEqual(3, customers.Count);
        }

Storing your events in an append manner on the filesystem is the easiest approach. Use of readonly objects simplifies things (easier to make sure that the data does not diverge). Perhaps a use-case for With?

The problem with using the filesystem directly is that it’s not ACID. However, you get ACID using OrigoDb.

Goto considered harmful

webmaster@gewalli.se (Oskar Gewalli) — Sun, 22 Feb 2015 23:30:11 +0000

Dijkstra wrote the following. I agree with his reasoning.

However, goto is useful in certain lower level domains.

The difficulty of reading code with goto statements is that it’s hard to follow the program flow.

The reasoning can also be applied to programming where you do not use goto.

Consider the following code:

someMethod(){
    // some state
    label 1:
        doStuff();
        goto end;
    label 2:
        if (condition){
            goto 1;
        }else{
            doOtherStuff();
            goto end;
        }
    label 3:
        doSomeDifferentThings();
        goto end;
...
    end:
}

How do you emulate the above code in a language without goto? You could use a while and a switch.

    someMethod(){
        // some state
        going = true;

        while(going){
          switch (goTo){
          case 1: going=false;
              doStuff(); break
          case 2: going=false;
            if (condition){
                  going=true; goTo=1;
            }else{
                doOtherStuff();
            }
            break;
          case 3: goin =false;
              doSomeDifferentThings();
            break;
        ...
          }
        }
    }

How do you emulate the above goto in a language with objects but without switch?

Object ThatDoesSomeThing{
    // some state
    public someMethod(){
        line_1();
        ...
    }

    private line_1(){
        doStuff();
    }
    private line_2(){
        if (condition){
                line_1();
        }else{
            doOtherStuff();
        }
    }
    private l_3(){
        doSomeDifferentThings();
    }
    ...
}

From stackoverflow:

    public someMethod(){
        int goTo=0;
        while(true){
          switch(goTo){
          case 0:
            doSomething();
            goTo = 1;
            continue;
          case 1:
            doSomethingElse();
            goTo = 2;
            continue;
          case 2:
            doSomethingDifferent();
            goTo = 0;
            continue;
          default:
            return;
          }
        }
    }

This translates into:

    Object ThatDoesSomeThing{
        // some state
        public someMethod(){
            line_0();
        }

        private line_0(){
            doSomething();
            line_1();
        }
        private line_1(){
            doSomethingElse();
            line_2();
        }
        private line_2(){
            doSomethingDifferent();
            line_1();
        }
    }

The main thing is the usage, not the keyword. Goto is an easy target because it’s famous for being bad. This is cargo cult programming. It’s easy to write an unreadable mess in many ways. In the above examples it might be due to the fact that it’s difficult to follow how the state of the context will be for any line.

The important thing to ask yourself is why is some code harder to read? How do you write readable code? How do you write good texts?

Looking at ORM a few years later

webmaster@gewalli.se (Oskar Gewalli) — Sun, 22 Feb 2015 11:20:38 +0000

When I first started using entity framework (one of the earlier versions) it was not ready yet (some serious performance issues). A few years later I hear good things about it from other developers.

In order to try to get a feel for how the state of entity persistance is today I’ve ported an old nhibernate test project to a test project for entity framework.

I initially had some trouble due to my choice of sql express to test the mappings. Once I managed to get EF to use sqlite the testing got much easier.

Both ORM’s support auto-mapping of POCO objects to the database entities. The fluent auto-mapping for NHibernate looks like it can figure out many to many relations.

The setup of NHibernate is a bit easier since you are not dependant on that web or app config factory settings. You get less confusion in the documentation for the NHibernate xml configuration. It looks like the people behind EF is aware of the problem and working on it.

Both frameworks can require some getting used to.

Note that both frameworks works well with fluent migrator to test against a sql-ish database. One way of getting a nice development flow could perhaps be to use sqlite on dev machines for integration testing and letting the CI machine run the tests against what you’re using on production.

Why global installation of packages?

webmaster@gewalli.se (Oskar Gewalli) — Sat, 07 Feb 2015 07:35:37 +0000

What do I mean when I say?

System level package / Global package: Package installed so that it is available for every program on the system
Local package: Package installed into sub folder of project and only available to that project

Timeline

This is just a sampling of when different packaging solutions appeared. My impression is that the first solutions where focused on system level package management. RubyGems and similar package managers came out of that context. I don’t know if it’s a coincidence but it looks like local package management tools started to appear first around 2010.

Global installation of packages probably dates back to system level packages. Since disk space is cheaper, there is no longer the same need to share packages among programs and services.

Render xml as html

webmaster@gewalli.se (Oskar Gewalli) — Sat, 07 Feb 2015 07:20:01 +0000

Sometimes when working with xml you have more information in the xml file than you know what to do with in your code. It could be new tags or attributes. A way to use the xml you have not anticipated in your code. A simple way to present that information to the user (so that they can inform you when you’re doing wrong) is to apply a general xml stylesheet.

First part of the trick is to tell xml that you intend to output html:

<xsl:output ... method="html" omit-xml-declaration="yes"/>

Next step is to create a root html (in order to be able to add style and scripts). In order to apply the rest o our xsl templates we need to apply the other templates:

<xsl:template match="/">
<html>
...
    <xsl:apply-templates/>
</html>

To render all the child-nodes we do the following:

<xsl:template match="node()">
...

Any attributes? By using @* we can pick out the xml attributes. If they show different intent it could be that we want to style them differently.

<xsl:if test="@*">

Any child nodes?

<xsl:if test="node()">

One thing you could do is to pass along the nesting as a parameter to the templates. Then you can add a different class to depending on how deep the element is.

If you want to call a template with parameters:

<xsl:call-template name="some-template">
    <xsl:with-param name="value" select="."/>
</xsl:call-template>

<xsl:template name="some-template">
    <xsl:param name="value"/>
    ...

Xml style sheets are a bit weird to work with. They should probably be avoided for many tasks. However, for some problems they can be quite useful.

Short talk about source control

webmaster@gewalli.se (Oskar Gewalli) — Sun, 30 Nov 2014 11:30:35 +0000

This is a rewrite of an earlier talk about source control.

What is source control? And why use it?

In order to track changes of your code you use source control. In the simplest setup, source control answers what your source code looked like before you accidentally broke but it looked like it was working.

In a similar sense you might want to store data about your customers in order to find it again. Who had a sale with customer A?

Why use Sourcesafe (or CVS)?

If you are a single dev.

Why not?

Technical answer:

Does the single file based database accessed using file system sound scary when hosted on network share? It is!
Sourcesafe is not supported anymore.

Less technical answer:

Problem when working more than one person.

Why use SVN (or similar with revision history only on server)?

The dev team are familiar with SVN and have an effective work flow in that tool.

Why not use SVN?

When you branch. It’s actually simpler to merge using git svn … but that adds the additional complexity of having to use git svn with svn. This can be ok if someone on the team can be assigned as an integrator that does the merges.

Why use Git (or mercurial et.c.)?

Can be an easy Sourcesafe like experience if you are a single developer. If you have lots of developer, it’s easier to work together using git.

When mentioning git, you cant avoid mentioning github. Git is version control. Github is hosting for that version control. Github has added some nice features as well.

Distributed source control?

Not connected to the corporate network at the moment? Do you work on a laptop (not connected to internet 100% of the time)? Do you like to continue browse the history and create commits? If yes, then consider using a dcvs.

Open source and github

Many open source projects have migrated over to github since it enables easier collaboration with untrusted developers. The mechanism at work is called Pull request.

In many closed source project you pay for support and eventual bug fixes. In an open source project you pay with developer time. Sometimes you have a mixed model (you pay for fixes and support but it’s open source).

Fork!

Any public library hosted on github can be forked if you have a github account.

Why is this sweet? Most of the time the devs behind the library does not care that much about fixing stuff relevant for you. If you fork the library and fix the problem you can send a Pull request with the changes. They might be rejected. They might be ignored. If you are lucky the devs will give you hints to improve the code (this improves the code and your skills).

Usability of git?

The command line version of git can be a bit complicated. However, when used by a single dev you can use Github for windows in order to work in a manner similar to Sourcesafe (simple gui).

A more advanced gui is the Atlassian SourceTree.

Hosting your code

There are several providers of git hosting. The ones I’ve been using are the following:

Github
TFS
BitBucket
CodePlex
Versioning for development code on Dropbox?

But there are several others. You could also self host with Gitblit or GitLab or others.

Installing build server

webmaster@gewalli.se (Oskar Gewalli) — Fri, 28 Nov 2014 09:11:37 +0000

So, for the N-th time I’m helping out by configuring a build server. This time it was a bit different since we where restoring an old build teamcity server and agent from a crashed server.

There where some gotchas as usual.

In order to restore teamcity we had to do some digging to find the data directory .

TeamCity data

If you look in the “teamcity-server.log” you can find how the server was started last time. From that we could conclude the old TEAMCITY_DATA_PATH. We took that directory and zip:ed it. Minus the cache, since windows zip freaked out about one the name of one of our coworkers: Michał Łusiak.

We made sure to set the system Environment variable:

TEAMCITY_DATA_PATH

to where we want teamcity to keep the data.

SQL server

From the teamcity configuration we could deduce how it connected to SQL server.

SQL server was installed with only database engine and management studio.

When SQL was installed we only enabled windows authentication. Since teamcity connects through a tcp port, it started complaining about not being able to connect even though we had attached the old database added the expected user.

If we tried to connect the same was as teamcity through the command line tool “sqlcmd” we could more quickly iterate over the configuration. The first problem was that sqlcmd could not connect to the expected port. In SQL Server configuration manager we had to enter the sub section “SQL server network configuration” and enable tcp. This got us a bit further. Now our problem was that the user could not connect with the user. We had created a user under main security in SQL server with the correct password. Problem was that the attached database still had a user with the same name. We removed that user and did “associate user with database”. We also needed to make sure that that user had the same permissions set as before.

Ok, so now we could log on to teamcity using our old accounts. Great!

The builds are failing because teamcity could not access the source control!

Easy fix, just need to add the private ssh keys to the correct folder so that teamcity can access private repositories.

Msbuild

So a lot of the builds where failing because msbuild was not installed. How do you install msbuild without visual studio? Turns out stackoverflow knows the answer: http://stackoverflow.com/questions/2567018/installing-msbuild-4-0-without-visual-studio-2010

You need the SDK.

I noticed that some of the builds where failing due to missing AL.exe (assembly linker). I installed the SDK associated with the version of windows installed Win2008. I forgot to check the error message closer. It was complaining about not finding “v8.0A” if you look at the error message. I had installed an earlier version if you look at the registry settings.

C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(2863,5): error MSB3086:
  Task could not find "AL.exe" using the SdkToolsPath "" or the registry key "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.0A\WinSDK-NetFx40Tools-x86". Make sure the SdkToolsPath is set and the tool exists in the correct processor specific location under the SdkToolsPath and that the Microsoft Windows SDK is installed

So I needed to install another SDK, the one called “Windows 8 SDK”.

The names of the SDK’s are a bit confusing.

Another project tried to build a web site. It was failing due to missing an imported msbuild targets file.

WebSite.csproj(190,3): error MSB4019: The imported project "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\WebApplications\Microsoft.WebApplication.targets"
 was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

The common solution here is to copy the targets for web applications from a developer machine in order to get these targets. They really should be part of a nuget package or available as a separate download.

Next day

Next day we ran into trouble with using port 80 for teamcity. Probably because the machine had slept during the night ;) . Turns out “system” is binding on port 80:

netstat -anbo  | where {$_.contains("80")}

When we searched to find some answer we found stackoverflow. Turns out “World Wide Web Publishing Service” is listening on port 80. Why? It looks like it installs when adding “.net 3.5.1” as a feature to windows. We removed the web server role.

Thoughts on using knockout for the last years

webmaster@gewalli.se (Oskar Gewalli) — Tue, 18 Nov 2014 20:11:43 +0000

It feels very much like the next step after jquery+templates. The nice part is that this enables you to choose your own strategy. For instance only using small amounts of code and regular forms. It requires a lot of decisions about structure decided already in a lot of js mvvm frameworks. These frameworks look like rails for client-side coding. Great for certain types of problems, but requires learning how to use it. Like always, if you can accept the limitations you get a boon. The big problem is that it’s hard to know such things in advance. For small apps jquery might be best. For larger single page like applications you should evaluate the larger frameworks.

Having a lot of knockout computed observables requires that you dig down into the Ko code base. This will probably improve with the maturity of Ko. A lot of computed observables can be created implicitly using a function in the view. On the knock me out blog there is some required reading Cleaning Up After Yourself in Knockout.js.

There might be improvements of IOC in JavaScript. Dispose and life cycle could be very useful for some single page apps. For knockout this amplified if you need long living objects. Need to evaluate the different IOC containers out there. Problem is that there is no dispose only a delete in plain JavaScript (how do you clean up from X?).

Having a proper url simplifies a lot of things. You have to do a lot of complicated code when your user interface creates a view that can’t be navigated to in the ordinary sense. Adding things like filter or navigation context can help you avoid hard to answer customer answers why certain things won’t work. Having the routing primarily server side can also simplify testing.

Bundler uses man files

webmaster@gewalli.se (Oskar Gewalli) — Thu, 26 Jun 2014 16:13:24 +0000

Turns out bundler does have documentation related to Bundler.with_clean_env. You just have to read the bundler.io man pages or the man pages for bundler exec.

Bundler.with_clean_env

webmaster@gewalli.se (Oskar Gewalli) — Wed, 25 Jun 2014 05:36:52 +0000

Using ruby is kind of easy in the beginning. We started by having a folder of loose scripts. This turned a bit messy after a while. By moving these to separate gems we got some needed structure. Many of the programmers complained about globally installed gems (i.e. our local gems). In order to get a predictable executing environment we started to leverage bundler more and more. Turns out that there is a gotcha: Executing a bundle exec script inside a bundle exec script. The inside script will inherit the environment from the outer script. In order to avoid this you can use:

Bundler.with_clean_env do
    ...
end

This method is not documented, but it really should be (I’ll do it if I find time).

Big news for .net

webmaster@gewalli.se (Oskar Gewalli) — Wed, 09 Apr 2014 22:39:00 +0000

Big news. Microsoft releases the next C# and vb.net compiler as open source. Miquel de Icaza speaks of making it work on mono as well. It’s licensed under Apache license.

Memory management in knockout

webmaster@gewalli.se (Oskar Gewalli) — Fri, 21 Feb 2014 20:21:50 +0000

Explicit use of computed?

What does a computed expression like this:

this.isBusy = ko.computed(function() {
    return self._isBusy() || self._queryIsBusy();
});

have to do with

<div data-bind="visible: _isBusy() || _queryIsBusy()" ></div>

Internally knockout creates a computed out of an expression in a binding handler. Ko does not parse the expression in the html above to register the variables _isBusy or _queryIsBusy.

Consequences

What does this mean for your code?

If you have something below but want to move the logic to your class from the view:

<div data-bind="visible: _isBusy() || _queryIsBusy()" ></div>

You have two main options:

Option 1

Object.defineProperty(self, 'isBusy', {
    get: function() {
           return _isBusy() || _queryIsBusy();
    }
});

<div data-bind="visible:isBusy" ></div>

Option 2

this.isBusy = function() {
    return _isBusy() || _queryIsBusy();
};

<div data-bind="visible:isBusy()" ></div>

Notice ‘()’

Life cycle

How long does the computed that knockout creates live?

A bit simplified if you read the code for KO you find that knockout makes new computed (internally dependent observable) with option: disposeWhenNodeIsRemoved This means that knockout will remove the list of references to observables that the computed depends on when the DOM-node that the computed is associated with have been removed.

When is it safe to use a computed?

When you know the life span of the computed you have created.

Why is it generally bad to use computed willy nilly

It is generally hard to reason about when they will be disposed. It is easy to access exposed ko.computed from different objects in the system with different life span.

Self and the gc

webmaster@gewalli.se (Oskar Gewalli) — Thu, 06 Feb 2014 17:59:00 +0000

For many projects I’ve been using the pattern:

    ns.CtorFunction = function(){
        var self = this;
        this.doStuff = function(){
            // do stuff and use self in order to get the object variables
        }
    }

Turns out that this pattern is good for small sites with jQuery (that makes use of this in other ways). For larger code bases where you have lots of code this construction sometimes makes it difficult for the garbage collector. If you have a large single page app, then this might be an issue.

What other patterns are available?

For a knockout js site you could use the pattern:

    ns.CtorFunction = function(){
        this.doStuff = function(){
            // do stuff and use this in order to get the object variables
        }
    }

Knockout sends the appropriate context to callbacks from the view (for instance the DOM click handlers). This makes the above construction work mostly fine. This depends of course of the scope of the knockout view (changed by “with”).

This might not be true for every app. My coworker Peter found that it depends on the amount of objects. It might be that the browser (chrome) have some limits to the amount of graph traversal that is allowed to identify entities to be garbage collected.

Writing rubyish C# or porting ruby code to C#

webmaster@gewalli.se (Oskar Gewalli) — Sun, 27 Oct 2013 14:41:00 +0000

Since I liked the ruby version for parsing phone numbers, I ported it to C#. In order to avoid having to rewrite to much I did the following:

To begin with, the entire project was using ruby name convention. This simplifies things.
Whenever there is a missing method where the behavior of the method is not the same as the C# one, I added an extension method. For instance I implemented a variant of gsub. The behavior of this gsub uses $1 for matching groups (since it’s more natural to remove a bit of code than make a full port).
I tried to translate different ruby constructs into C#. For instance the lazy pattern found in the library @variable ||= init_variable I had to replace with _variable ?? (_variable=InitVariable())
After all the tests became green and I had checked in the initial version, it was quite simple to change name convention using ReSharper.

The finished version is available on github

Using values in expression

webmaster@gewalli.se (Oskar Gewalli) — Sun, 20 Oct 2013 09:47:00 +0000

I’ve done some small tests while writing the library to handle readonly objects in C#. Turns out that getting the value of a sub expression can be costly (since you have to compile the expression and execute). When just setting one property there can be a performance benefit of using the following syntax:

new MyClass(1, “2”).With(m => m.MyProperty, 3)

There might be some way of optimizing the code in order to get better performance for the form:

new MyClass(1, “2”).With(m => m.MyProperty == 3)

If there are only a few usages then the performance hit is probably negligible.

Using readonly objects in C#

webmaster@gewalli.se (Oskar Gewalli) — Fri, 18 Oct 2013 17:39:00 +0000

One thing I like about F# is the ability to use readonly objects in a way that does not require a lot of code. I’ve done a small lib inspired by this ability of F#: with

The syntax is a bit special since I assume that the properties have a private set.

versions prior to 5

myobject.With(m => m.Property1 == 3 && m.Property2 == "3")

With 5+

using With;
...
public class SomeClass
{
    private static readonly IPreparedCopy<MyClass, int, string> PreparedCopy =
        Prepare.Copy<MyClass, int, string>((m,v1,v2) => m.Property1 == v1 && m.Property2 == v2);
    public async Task<MyClass> Handle()
    {
        // fetch instance of MyClass, say:
        var myObject = await _storage.Load(someId);
        // change the name of that customer:
        var copy = PreparedCopy.Copy(myObject, 3, "3");
        // ...
    }
}

The reason for this style of syntax is due to the fact that compiling expressions are potentially expensive, why you’d want to have a similar pattern as compiled regex.

Writing build scripts in C#

webmaster@gewalli.se (Oskar Gewalli) — Sun, 13 Oct 2013 06:42:00 +0000

I went to leetspeak yesterday. Justin Rusbatch had a talk about scriptcs. In that talk he mentioned that several people wanted to write build scripts in C# and thought that it would be nice to use scriptcs to do that. I started trying out to write build scripts in C#. Turns out that many of the classes that are available in .net are not supposed to be used by users of .net. It’s cleaner and simpler to execute the command line tools from C#. I started writing some code to try it out. I called it shake.

Problem running NuGet on Ubuntu

webmaster@gewalli.se (Oskar Gewalli) — Sun, 06 Oct 2013 21:19:00 +0000

I had some trouble running NuGet on ubuntu. I’m running an Ubuntu server. Turns out that you need to install certificates:

mozroots –import –sync

Otherwise NuGet will fail to decode the results when fetching packages over https.

Skånetrafiken

webmaster@gewalli.se (Oskar Gewalli) — Sat, 21 Sep 2013 18:41:00 +0000

I dusted off some old code that I’ve written to query the API provided by Skånetrafiken. I’ve also published it to rubygems. It’s quite nice to work with soap using ruby.

Isop infers parameters

webmaster@gewalli.se (Oskar Gewalli) — Sat, 21 Sep 2013 07:13:00 +0000

Command line tools should do what people expect. It’s a bit chaotic to navigate how to parse command line parameters since there are many conventions.

A single - to indicate a short parameter, that is: -a
Two – to indicate a long parameter, that is: –alpha
A single / to indicate a parameter, that is: /alpha
A single - to indicate a parameter, that is: -alpha
A parameter followed by a space and then a value for that parameter: /alpha value
A parameter followed by a = and then a value for that parameter: –alpha=value
The position of the parameter (the ordinal position) to indicate simply a value

I’ve added code to try to infer value for a parameter using it’s ordinal position when there are no ordinary parameters present before the inferred parameter value.

This simply means that it should be possible to write:

my.exe --param1=something --param2=someother

and also:

my.exe something someother

What's wrong with spreadsheets?

webmaster@gewalli.se (Oskar Gewalli) — Tue, 10 Sep 2013 18:23:00 +0000

Whats good about spreadsheets?

Spreadsheet applications offers a lot of usability. The concept of using a grid of variables that are easy to reference is a surprisingly powerful concept.

Many applications can be prototyped in spread sheet applications. I use google drive in order to have a powerful and flexible time reporting solution. I quickly realized that in order to have a software that exceeds these benefits, I would have to spend a lot of time writing it.

What can be improved?

A few days ago I was about to do something that is usually simple in code, but turned out to be hard in excel. The functions available have not evolved very much since the introduction of spreadsheet applications. This is probably due to the fact that the people expected to work with these type of applications are not expected to learn new concepts (Excel looks very different from the early spreadsheet programs). During that time, commonly used programming languages have evolved. Perhaps it’s time to put the lessons learned in programming to use in spreadsheet applications.

There are some difficulties in applying concepts from programming to spreadsheets. There is not a natural way of assignment for functions that return multiple results, structs or objects. This puts a limit on what type of functions can be used. However, I do believe that it’s possible to use functions that return a single string or number variable to increase the usefulness of the concept.

Refactoring in ruby

webmaster@gewalli.se (Oskar Gewalli) — Wed, 04 Sep 2013 20:07:00 +0000

We have written a lot of ruby code in the project I’m currently working in. There is a lot of code to help manage, build and schedule the program. We started out simple enough. Just one folder with helper scripts. After 2 years we have built up support for a wide variety of tasks.

Command line tools

We have used and abused $:.unshift File.dirname(__FILE__) and similar constructs far to much. This makes it hard to get a good overview of dependencies between files.

In order to get islands of related code we have turned to making local only gems. The gem structure helps to cut down on weird manipulations of the load path. It’s easier to get an overview by having related files and their tests in a separate folder.

Bundler

Bundler turns out to have a lot of nice tasks (a coworker told me about it): require 'bundler/gem_tasks'

Looking at the bundler code helps when using the GemHelper class. If we look at gem_tasks.rb we find:

require 'bundler/gem_helper' 
Bundler::GemHelper.install_tasks

The standard format for gemspecs usually make use of git ls-files. We got a lot of error messages from bundler and switched to use Dir.glob.

Rails app turns into gem

I’ve written a rails-app for a specific task. I started out with having most of the code in models, helpers in the rails app folder. This turned a bit sour after a while. In order to alleviate the code smell, I started moving the functionality into the lib folder. It turned out that we wanted to use the code in a different setting than as a web app. I moved the relevant tests and code from the lib folder into a new package/gem and added a dependency on this gem from the rails app.

We have since stopped maintaining the rails app, and it has been removed.

National clouds

webmaster@gewalli.se (Oskar Gewalli) — Sat, 06 Jul 2013 20:15:00 +0000

We first heard of the Australian government getting fallout from the US because of voiced concerns about using offshore cloud providers. Some technologists I have talked with thought it obvious that the US government would use available means to get accurate information given the obsolesce of certain infrastructure. This might spell an increased demand for local alternatives for business software. Another way is to start deploying software in a more security conscious manner. For instance encrypting the stored information so that the cost of espionage increases.

Listening to traffic

webmaster@gewalli.se (Oskar Gewalli) — Mon, 20 May 2013 10:58:00 +0000

While developing a web app, it’s easy to inspect the traffic in the browser (chrome, IE and firefox have all the ability to show traffic related to the page you are debugging). I recently refactored a lib to remove a library and use a raw web-request. This was a move to remove dependencies. In my tests I added machine name instead of localhost and got the traffic in Fiddler. An easy way to remove the abstraction and start accessing the real json api.

Test your update scripts

webmaster@gewalli.se (Oskar Gewalli) — Thu, 13 Dec 2012 15:14:00 +0000

This is a lesson I should have learned. It comes back to bite me and my coworkers ever so often. Running a script to update data without test is always quick and dirty. SSIS is a wonderful tool, but there are fundamental flaws to this tool as it’s really hard to test SSIS packages. The beauty of using ruby or some other language (powershell, python, et.c.) is that you can write tests for the code. So, when you’re doing yourself the favor of using ruby, write tests!

Writers and readers

webmaster@gewalli.se (Oskar Gewalli) — Tue, 04 Dec 2012 21:51:00 +0000

Reading source code is hard. One way of dealing with this is rewriting what is not understood. This can help focus on some parts of the code, but can be detrimental since it’s easy to overlook functionality in files that are not touched or files that look too complex (throwing away functionality). One way of doing this is to create a fork to rewrite one, in order to be able to throw away the changes made essentially to read the code.

It’s usually harder to read code than to write it. Trying to read a 700 line method can be to hard, thus it may need some therapeutic refactoring in reduce the components to pieces that can be understood.

Among several programmers, one reason why there is a tug o war between them can be because of non orthogonal and different best practices: They keep rewriting each others code because they have different ideas on what tradeoffs to prefer.

Type matching in C#

webmaster@gewalli.se (Oskar Gewalli) — Thu, 22 Nov 2012 18:27:00 +0000

I was thinking about how nice you have it in some languages where you can do a case statement for types. Thing is, we have a lot of compiler goodness in C# already.

By being explicit about the type and using type inference we can have a generic Case statement of a maximum length (depends on how many if else you think are ok to add).

I find it pretty sweet to write code like this:

TypeMatch.Case(exception,
  (GnarlyType e) => { HandleGnarly(e); },
  (FuncyType e1) => { HandleFunky(e1); },
  () => { Explode(); });

Instead of:

var e = exception as GnarlyType;
if (e!=null){ HandleGnarly(e); }
else {
  var e1 = exception as FunkyType;
  if (e1!=null){
    HandleFunky(e1);
  }else{
    Explode();
  }
}

Small note

This type of feature is now part of C# proper as:

switch (exception)
{
  case GnarlyType e: HandleGnarly(e); break;
  case FunkyType e1: HandleFunky(e1); break;
  default: Explode(); break;
}

Debugging fuzzy coding

webmaster@gewalli.se (Oskar Gewalli) — Sat, 10 Nov 2012 17:11:00 +0000

As a programmer it is easy to adopt a view of the world in light of our experience. The workings of the world starts to look as something out of the matrix. This has some funny consequences: While listening to a person reasoning about the complicated events today from the point of view of famous thinkers; You can get the feeling of hearing a formalized thought pattern executed by a very complex machine. This sort of meta thinking naturally can be attributed to the formalization of execution that you work with, i.e. it is a view of programmable minds from the view of a mind programmed by thinking about programming.

This kind of thinking is useless unless it can be applied.

In the terms of a programmer: We are looking into how to debug the domain. The most important aspect of getting to know a domain is learning the language of that domain. The meaning of words inside that discussion context.

How do we reduce the task to something manageable? Simply learning all the language and statements associated with a given domain a huge undertaking. What is it that we want to learn? One of the interesting aspect of the domain is the grammar and the macro expansion of that domain. If we apply this to the world of programming, what kind of form of statements and what kind macros are we thinking in?

Most programmers work with the following concepts (I’m trying to simplify it): methods, classes (and instances of the classes), variables The most important execution constructs are: if, for

Naming methods, classes and variables turns the execution into a meaningful flow. Mostly we are working with details surrounding execution of our concepts. A programmer is working a lot with detail management (much like a weaver of a tapestry have to manage the the threads in order to get a working picture). A very serious problem is that it’s hard to find the right level of detail and abstraction to work with. The concepts above can be used to create detail at arbitrary levels. Kind of like Lego versus Duplo.

For the most part, many tend to prefer immutable constructs: Specified classes that do not change their behavior after initialization. This greatly simplifies reasoning.

Since programmers are working so much with abstractions, we tend to overdo abstraction.

What kind of conclusion can I draw from this? I’m not sure. It reads like the exact opposite of statistics. Mostly we program in predictable ways that get unpredictable due to the complexity rather than randomness of the input. Do we expect people to behave in this manner? Well, they sometimes behave similarly at a cursory glance. The big question would be what kind of programming we would write for people?

The code I wrote is a mess

webmaster@gewalli.se (Oskar Gewalli) — Tue, 06 Nov 2012 23:14:00 +0000

Many times when you return to code that you have written you will swear and be annoyed with your lack of foresight.

We have been using a library since a while back. There is a definite strength to having a simple way of exposing actions from a console application. The code for the library and runner Isop is a mess. A mess designed to avoid having to code a messy console application in order to do quick interactions with your code (.net code to be specific). The focus of the lib has always been to cater to the needs of business applications. I’ve not spent time cleaning up the code since it satisfies the criteria: If it works, why fix it?

It annoys me that I don’t have the time and energy to write beautiful code. This instinct is both healthy and dangerous. It helps you create maintainable applications, but it can also get you mired down into polishing. Ultimately it goes down to the fact that different structure caters to different intent and intent often changes (thus the unpleasant hindsight). Sometimes it’s that you had something simple to begin with, but as you fill in the gaps it becomes complicated.

What I’m trying to say is that it’s really hard to avoid the problem of complicated code or code that does not fit your intent. What you need is to refactor. This is the skill to use.

Why should you make your new developers use ruby?

webmaster@gewalli.se (Oskar Gewalli) — Mon, 08 Oct 2012 22:42:00 +0000

The impression I’ve gotten from working with ruby is that it’s generally fun to work with. One of my coworkers who has about a year of experience told me:

You become happy from working with ruby.

Also, for many tasks it can provide a easier experience than some of the alternatives. Consider the tasks:

Manage builds or server
Simple web app

Tools like rake and sinatra are simple enough to help green developers complete the task without drowning in the complexity.

It’s the same reasoning behind webmatrix. In webmatrix there is a lot less focus on unit testing. It’s a slightly different perspective, but still centered around simple fun stuff.

Dark-room development

webmaster@gewalli.se (Oskar Gewalli) — Wed, 03 Oct 2012 06:24:00 +0000

One of the fears is that the days of PC is considered over. That we there will be a massive shift towards a reduction of the interface. Some have pointed out that the start screen on Windows 8: It’s Almost Not Terrible. Gartner sums it up in a word: Bad for desktops (however he also says that it’s very fine on touch device).

This all sounds very odd. However, the negativity mostly centered around how to use the start screen on a desktop might not be such a big deal. Hanselman among others assures that you just have to learn the new keyboard shortcuts and that it’s just a matter of getting used to. Our IT-boss (a small company so he does a lot of the IT-operations) tells me that a power user will hardly use the start screen.

As long as the two separate entities desktop and xbox/tablet interface live side by side there might not be cause for alarm for programmers using multiple windows side by side (editor, library documentation, tests, powershell, command prompt et.c.). Other programmers may in fact welcome the one window approach. Some have already used visual studio as the one program to do all things. Also for writers of text there are a few programs for windows and mac that focus on full-screen to avoid distractions and get things done. It’s exciting to see where Windows 8 goes.

TypeScript

webmaster@gewalli.se (Oskar Gewalli) — Tue, 02 Oct 2012 19:38:00 +0000

Coffeescript has been heralded as a way to avoid some of the nastiness of JavaScript. Now Anders Hejlsberg introduces a new addition to the story: TypeScript. I read this from a pal that linked to a blog post. The geist of the post is a slight positive note, but the author notes that IntelliSense is missing for editors other than Visual Studio. If the license is good (it’s in Apache license), the language additions are relevant, there might be people that get some of it to work in another editor.

Funky xml log

webmaster@gewalli.se (Oskar Gewalli) — Tue, 25 Sep 2012 06:13:00 +0000

I had some trouble where a simple parser would take a bit of time. Turns out that there were many ways of improving this performance. I rewrote it to use the xml reader instead to be able to do these improvements. Some funky bugs got squished. The reader is a bit low level compared to niceties such as Nokogiri (in hindsight, maybe I should have been using). By using this class I could avoid some funkiness of XmlDocument having to cope with weird xml at the expense of having less appealing code: LogEntryParser.

Whats right with windows 8?

webmaster@gewalli.se (Oskar Gewalli) — Mon, 24 Sep 2012 17:42:00 +0000

Imagine the user experience of a gaming console. It’s very focused on a selected few tasks: starting games, playing movies… An ipad is conceptually a similar device. You can surf, run certain applications, watch movies, listen to music. The user interface is simple. Most users have no need for the fuss.

To imagine an operating system where you focus on what normal users want to do, you start looking in the windows 8 direction. The start screen reminds of the Xbox.

I got skeptical and confused when I tried windows 8, looked at videos about it. The reason why it felt wrong: Windows 8 is not intended for me. I’m not part of the 95% that abhors complexity. Ordinary user dislike the “to much information” syndrome present for most desktop operating systems.

I have a suspicion that it’s not there yet. Maybe when windows 8.3 or windows 9 is released, my mom can use it without getting frustrated. There are just a few things she wants to do: Surf the web, pay the bills, read her email, manage spreadsheets, write documents, manage her photos and keep the gigabytes of photos backed up.

Always attaching debugger when starting a program in windows

webmaster@gewalli.se (Oskar Gewalli) — Mon, 24 Sep 2012 09:20:00 +0000

So, Im using Process Monitor to hunt down some permission or anti virus problems in windows. I notice a query to a registry key “Image File Execution Options”. I get curious. Turns out that you can turn on a debugger when starting a program. Someone used it to control a virus outbreak, but it could be handy for other things as well.

Visual studio 2012 all caps

webmaster@gewalli.se (Oskar Gewalli) — Thu, 16 Aug 2012 15:43:00 +0000

We are waiting for VS2012. What will impact work experience most will probably be the user interface changes as described in: Visual Studio 11 User Interface Updates Coming in RC

One thing I don’t understand is the preference for all caps. The interface is riddles with them. However Richard Banks describes how-to prevent VS2012 ALL CAPS menus, so there might be hope yet.

Spending hours on trying to figure out why it does not check

webmaster@gewalli.se (Oskar Gewalli) — Tue, 31 Jul 2012 07:36:00 +0000

Yesterday I spent hours trying to figure out why a checkbox did not get checked (visually). When debugging the knockout code, it was clear that the DOM elements got checked. They did not update visually until another checkbox got checked. I tried other methods of checking the checkbox (both setAttribute and jquery) to no affect. What was even weirder is that it works the same in both chrome and IE.

To reproduce, click on the first checkbox and then the second in this fiddle.

I got the question if I had a click event handler from steal my code.

Turns out that the reason for this behavior is that I had a click event handler on a parent DOM element. By having the click handler return true the problem was resolved.

I still don’t understand why it works like this.

Is philosophy relevant for programmers?

webmaster@gewalli.se (Oskar Gewalli) — Tue, 24 Jul 2012 18:30:00 +0000

Many programmers and engineers I’ve met say when asked about philosophy:

I don’t really care for it. It’s obsolete because of science.

Could it be that it’s the engineering point of view? The view that some scientist (tool maker) cooks up the tools that the engineers use? Thus: truth and reality is simply handed down from a more trustworthy source.

Science picks up the tab

Science ( trademark of modern civilization ) has thus replaced philosophy ( funny thoughts by bearded old men ). However, the philosophy taught in schools is mostly focused on aggregating history into manageable chunks. It’s simplified history about thoughts spanning circa 2500 years.

I like the below explanation of philosophy:

Anthony Quinton, in T. Honderich (ed.), The Oxford Companion to Philosophy (Oxford University Press, 1995), p. 666: “Philosophy is rationally critical thinking, of a more or less systematic kind about the general nature of the world (metaphysics or theory of existence), the justification of belief (epistemology or theory of knowledge), and the conduct of life (ethics or theory of value). Each of the three elements in this list has a non-philosophical counterpart, from which it is distinguished by its explicitly rational and critical way of proceeding and by its systematic nature. Everyone has some general conception of the nature of the world in which they live and of their place in it. Metaphysics replaces the underlying assumptions embodied in such a conception with a rational and organized body of beliefs about the world as a whole. Everyone has occasion to doubt and question beliefs, their own or those of others, with more or less success and without any theory of what they are doing. Epistemology seeks by argument to make explicit the rules of correct belief formation. Everyone governs their conduct by directing it to desired or valued ends. Ethics, or moral philosophy, in its most inclusive sense, seeks to articulate, in rationally systematic form, the rules or principles involved.”

From the wikipedia article about philosophy

Do you really need it?

Thing is, you still need to think on your own. You still need to determine what you want to do with your life. What reality or world view you want to have. As programmers we use language and abstractions as tools. In order to use these in a “constructive way” (not jiggle things randomly until they work), you need to think and learn.

Evaluating software

In many cases you need to evaluate things on your own. You will encounter different entities that try to get you to use different tools and different libraries. Or will you blindly let companies shove their solutions down your throat?

What kind of shift of reality do you experience when you change environment? For instance change:

Programming language
Web framework
Between desktop and web programming
Between server and client programming

When shifting between these environments, it’s also a shift in how you solve problems. Different programmers value different writing experiences. Compare the static typing of java to write client applications in Google Web Toolkit to scripting using JavaScript. It’s a reflection of how you think about programming.

How do you evaluate a library? If it’s commercial, you try to guess how long it will be maintained by the company making it. If it’s open source, you try to ascertain how well maintained it is. If it is currently maintained et.c.. You try to guess the quality of the library. Also there is such aspects as cost and license. Can you afford to use the library? Does the library have a licence so that you can use it together with your proprietary code base?

You are not a simple consumer of tools

As a programmer, you should be willing to write your own library if that’s what you need. You should be able to evaluate what tools to use when doing your work. Programming is the act of creating text that can be used as tools. You are a toolmaker and a tool consumer. Builder and toolmaker. You are an author and a reader. It helps to think.

Web versus native app login

webmaster@gewalli.se (Oskar Gewalli) — Fri, 13 Jul 2012 14:53:00 +0000

What are the most important differences between native applications and web applications (for mail, rss et.c.) when it comes to login?

Native applications usually store login information. For web applications this is dependant on the browser in order to store login form. Depending on application, there can be days between resubmission of credentials.

For unix applications there are alternatives like ssh shared keys. It enables connection from one machine (login) to another. It would be interesting if this approach could be reused for browser to web server communication. That is: generating a shared key for a single browser/machine. If you could generate a one time use string from the phone company in order to anonymize the identity? The identity of the user would still be transparent to ISP and other infrastructure providers. The setup to use a site from a new machine/login would be a hassle. For single use scenario: Maybe one session tokens sent to the phone when you are at an internet cafe?

Currently the alternatives are reduced to form fillers and password managers. Some browsers implement these features.

Refusing to work with a technology or a tool

webmaster@gewalli.se (Oskar Gewalli) — Sun, 10 Jun 2012 12:21:00 +0000

Refusing?

As a developer, assume that you are given technology or tool X and told that you need to create Y using it. Sometime it’s just getting down to business and getting it done. Sometimes you realize that using said technology is a mistake. It could be that the technology isn’t appropriate for the task. It could be that there are simpler cheaper solutions (things that take less of your time). It could also be that you feel that it’s outside of your job: Like asking an accountant to do the floors.

What do you do?

What you feel as professional, refusal to waste money and resources, could be interpreted as unprofessional. Your refusal could be interpreted by your manager as behaving as a child: “Developers who act like children (refusing to use a tool mandated by a customer!) are not good developers no matter what their coding skills. …” The above answers sounds a bit childish: Since my parents refuse to let me eat candy, they are not good parents. Another answers deals with the role of a developer (slightly rewritten): “you hired developers and you are trying to get them to do something else”

If it’s not your job

A more experienced coworker told me that the best action is to endure. If you get to do more interesting things later on, then it could be worth hanging in there. If your manager starts to give you more of the same work, since you have done so well before, then it might be time to look for a new job. It could be that your skills as a developer are less valued than work with technology or tool X at that company.

Creating shoddy solutions?

Developers are usually driven to do a good job. I’ve done things I’m not proud of. I’ve helped create solutions that are shoddy because the underlying technology is wrong for the customer. But I’ve done it since the customer wanted it. I’ve tried to do the best job I can. I’ve informed my superiors about the problems. It could be that the shoddy solution is good enough for the customer. It could be that you realize that technology or tool X is really good when used specifically to do the things it’s designed for. You can then help customers use X when it’s appropriate.

Try to offer alternatives

It could be that the request to work with technology or tool X is simply the request to get work Y done. If that’s the case, then you can reason with your manager about not involving technical terms when talking about what needs to be done. It could be that a change of requirements can make the technology or tool a better fit. In that case, you should try to get your manager to steer the customer in the right direction.

What should you avoid?

Making it personal or emotional. This will only make it harder in the future to work together.
Refusing in front of others. This might require your manager to save face.
Talking about the technical reason to a non technical manager or customer. This will sound like an excuse to a non technical person. You should give an easy to understand summary.
Fooling yourself into believing that technology Y, that you have not tried yet, is better. It might be better in some aspects. It might also be that it’s got different problems.
Fooling yourself into believing that technology Z, that you usually use, is better. If you don’t know technology X, then an outright dismissal of it is without grounds. That said, you should be skeptical of any technology that you hear fearful rumours about.
Fooling yourself into believing that you can roll your own framework that does it better.
Fooling yourself into believing that you need a framework when all you need is a bit of code.

Technology and music

webmaster@gewalli.se (Oskar Gewalli) — Sun, 03 Jun 2012 19:08:00 +0000

We don’t even know how old instruments are

The earliest type of instrument is thought to be flutes. According to wikipedia this technology could be as old as 43,400 and 67,000 years. Simple devices for making rhythm where probably made: drums, rattles. But since they are usually composed of organic matter, they would decompose leaving little trace. Without knowing much I would guess that humans developed stringed instruments relatively early. My unscientific guess is that these kinds of instruments developed alongside bow and arrow. Making them perhaps 18,000 years old. Maybe even older. Or perhaps the use of instruments as a musical tool is more recent (maybe 9,000 years old). I’m very much colored by my perception of human artistic expression.

Metal working

As technology advanced, so would the potential for musical instruments advance. With metal working cymbals, sistrum, bells as well as gongs can be made. Trumpets could be made in metal to produce new sounds.

Hydraulic and pneumatic

The use of a mechanical power source enabled the hydraulic organ. The church organs must have been expensive technological marvel. It is a huge and impressive instrument. Probably one of the must awesome noisemakers before the invention of electric instruments.

Ca 18th century

The invention of the piano is around the late 17th and the beginning of 18th century. It’s an amazing technological advancement: A key operated string instrument.

The 19th century

One specific piece of technology are behind some of the brass instruments: Valves. A recent tech, developed in the 19th century. An older technique was to use keys to open holes in a wind instrument.

Early 20th century

In the 20th century electric instruments where developed. An excellent example of this is the electric guitar. The solid body electric guitar embodies the advent of modern industry. Produced with a wooden solid body and a bolt on neck together with magnetic pickup. It’s industrial design! A continuation of the old stringed instruments into the modern era.

An exciting development is the theremin. An instrument totally unlike older instruments. Two metal antenna to sense the position of the hands to control frequency and amplitude.

We have the telharmonium from just before the 20th century. It’s a musical instrument made with tone wheels. It is an instrument that was made to generate musical sound from an electric signal. The instrument never caught on since it required immense amounts of power and weighed almost 200 tons. It was an impressive technological achievement since it could reproduce the sound of common woodwind instruments. Hammond created an affordable instrument, using tone wheels, called the Hammond organ.

Later instruments

With digitally generated sound, there are huge possibilities. There are enormous possibilities for advancing the usability of tone/sound generation. Another important factor is the price of instruments. A simple flute is very cheap. More complex instruments like an electric guitar or a keyboard are very affordable instruments. A keyboard has excellent usability: The keys maps to the chromatic scale.

Github for windows

webmaster@gewalli.se (Oskar Gewalli) — Fri, 25 May 2012 12:22:00 +0000

We had a small problem with one of the open source components that we use. One of my coworkers had to make a small alteration to that component (he isn’t using git at work). The github for windows program makes interacting with github very easy on windows. All he had to do was to install the program (easy installation) and clone his fork. Really useful for this kind of work.

Regarding JavaScript namespaces

webmaster@gewalli.se (Oskar Gewalli) — Sat, 07 Apr 2012 07:22:00 +0000

There seems that the there is a shift from the Crockford style global variables to lower case namespaces or mixed case. Older libraries use the Crockford style all caps.

YUI
JSON

Probably this is influenced by Crockford’s thoughts about global variables

Newer libraries use a less screamy namespace name: jQuery, Prototype, Ext, dojo, ko

Domain analysis of an application database

webmaster@gewalli.se (Oskar Gewalli) — Sun, 18 Mar 2012 12:32:00 +0000

A relational database is not only a source of content. It can also be used as a mine for domain information related to domain of an application that uses that database. The schema, if it is well maintained, will give you an overview of the problem domain. If the schema is not well maintained, you can factor out the conventions used in that particular database for relations and create a patched schema with the real relations. As you generate a graph of the database you can start filtering out some of the details (important for business logic) but not for the big picture. Some of the things you can get from the graph will be trivial. It might also give you a better understanding of the big picture. By plotting this graph, you can get a overview helpful for identifying subdomains.

Maybe if we have a sufficiently large body of text describing the domain we can use a statistical approach to identify words relevant to the domain. The identification of these words will probably be easier if we have some larger body of text to use as for comparison of the frequencies of sentence construction words in that language (that is to filter out words like: ‘the’, ‘for’, ‘he’, ‘she’ et.c. in english).

By combining the above simple domain mining techniques and talking to people fluent in the domain you can get a shallow technical understanding of the domain. This can be helpful for understanding the details of the domain.

Project Roslyn

webmaster@gewalli.se (Oskar Gewalli) — Sat, 17 Mar 2012 10:57:00 +0000

In some settings you might need to access more of the language than available through reflection and System.Linq.Expressions. You might want to look into project Roslyn or NRefactory. These will let you access the C# parser. This can be useful in many settings. For instance, to refactor C#. To render the code to another language.

Language

webmaster@gewalli.se (Oskar Gewalli) — Thu, 08 Mar 2012 07:31:00 +0000

Make sure that the environment variable LANG is set to “us_EN.UTF-8”. Otherwise you might get the translated versions.

Chocolatey

webmaster@gewalli.se (Oskar Gewalli) — Tue, 28 Feb 2012 18:56:00 +0000

Thanks to Fervent Coder we have Chocolatey. It can be used to setup a development machine with new software. Installing java, notepad++ becomes as easy as:
cinst javaruntime

File upload

webmaster@gewalli.se (Oskar Gewalli) — Wed, 22 Feb 2012 20:00:00 +0000

The alternatives right now (2012-02-22):

Standard html: File select together in a form that is submitted to a page. (firefox, chrome, IE)
Iframe hack: File select together with hidden iframe used to upload file. (firefox, chrome, IE)
Flash (all flash enabled browsers)
FileReader (firefox, chrome)

Drag and drop together with FileReader and ajax post.
File select together with FileReader and ajax post.

When building ordinary web applications then the first alternative is definitely the best. It’s simple and works well. If you are building a more JavaScript intense application you are in a different mess. If you are supporting IE 7-9 then alternative 2 and 3 is what you are looking for. To enable the best experience you could sniff if the browser handles FileReader (since IE10 might support it) and enable a better experience in that case.

Note: I’ve only considered firefox, chrome and IE.

Continuous code coverage

webmaster@gewalli.se (Oskar Gewalli) — Sun, 19 Feb 2012 16:54:00 +0000

What if we could do a continuous code coverage? That is, while debugging and testing the app, storing the coverage information. A first version would consider lines that are not changed as covered if they have been executed during previous runs. This could be a sort of blunt tool to help isolate code that should be flagged either for removal, for code review, automatic or manual testing.

It should be simple enough to send information about code coverage when running JavaScript code. The challenge is to map the coverage information with source lines and keeping some sort of data structure that represents the code and coverage information that can be updated with the old coverage information form previous versions of the code.

Avoid batch files

webmaster@gewalli.se (Oskar Gewalli) — Fri, 17 Feb 2012 19:36:00 +0000

When programming on windows, you will sometimes need to execute batch files. It might seem like a good idea to do something simple in a batch file. This is the time you should realize that it it’s a slippery slope. Dont go there.

There are excellent alternatives! Powershell works well for scripting (as long as you accept the signing or set-execution-policy). Ruby is sweet for scripting. There are alternatives that will let simple things be simple.

Metro style revolution

webmaster@gewalli.se (Oskar Gewalli) — Sun, 29 Jan 2012 13:14:00 +0000

The more I think about the metro style, it sounds really revolutionary for microsoft design. I’ve started looking at how the new windows phone style looks. It looks more modern than the other themes for smart phones. Probably because it’s not been used as much.

I feel that windows aero was to bloated in visual space. It looks sweet on a machine with high enough resolution and screen space, but looks to bulky on other things. The default margins take to much space. The classic gray theme has been my preferred theme when working in windows. When working, it’s nice to have a theme that does not require attention.

So the conclusion is that I’m excited about the possible metro theme for windows.

Using backbone routes together with knockout

webmaster@gewalli.se (Oskar Gewalli) — Tue, 13 Dec 2011 11:00:00 +0000

We have Scott’s method. He uses backbone together with knockout. You might want to read the following google group posts. I like how you declare your setup on the app object. The problem with using only knockout for creating js app’s is that it only focused on view to modelview binding. Also Derick Bailey has made a comment about the curious idea that you choose either knockout or backbone.

MVVM Plus Conductor with Knockout js

webmaster@gewalli.se (Oskar Gewalli) — Thu, 01 Dec 2011 16:34:00 +0000

A better fit in the browser world for knockout than MVC is to have something like a controller. A class responsible for being an endpoint for routes. This is naturally coupled with the responsibility of changing between screens. Since controller is such a loaded word, lets simply call it conductor.

That gives you the following components: Model view, view-model and conductor. In order to implement the pattern you can use crossroads js or something similar.

That means that crossroads will route to different conductors that marries view-models and views and sends them off. When the view-model changes in such a way that it needs to update the url, it tells the conductor to change the url. Anyone can also ask a conductor to present an object (thus changing the url and updating the screen).

using underscore filter in IE 7 & 8

webmaster@gewalli.se (Oskar Gewalli) — Wed, 09 Nov 2011 20:00:00 +0000

Array.prototype.filter seems to be slightly off in IE7-8. It looks like they have added support for standard array functions. That is, it looks like there is a prototype for this in IE7-8 mode in IE9. The problem is that the filter prototype does not work exactly as expected in underscore.

If we look at the documentation for this method on msdn you find that this is probably a bug.

It is simple to patch the prototype implementation you find using F12 in IE. Otherwise you could use the prototype described in the stackoverflow post (i.e. Mozilla’s Array filter implementation): https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Array/filter

Press play on tape

webmaster@gewalli.se (Oskar Gewalli) — Mon, 17 Oct 2011 08:31:00 +0000

Lovely geekery when press play on tape plays cannon fodder with old game controllers. Why not monkey island :).

dotnet-console is now Isop

webmaster@gewalli.se (Oskar Gewalli) — Sat, 08 Oct 2011 21:28:00 +0000

I’ve renamed the dotnet-console project on github to isop. Isop is the swedish name for hyssop. Like any spice it is intended to give flavor to the development of command line apps in .net.

Using msysgit in cygwin

webmaster@gewalli.se (Oskar Gewalli) — Fri, 07 Oct 2011 09:21:00 +0000

There are some pitfalls. One for instance is that it starts the editor wrong when running msysgit in the cygwin shell. Naturally I tried specifying different paths and ended up with the following configuration of gitconfig:

c:\\cygwin\\home\\wallymathieu\\.gitconfig

[core]
  editor = "\\"/c/Program Files (x86)/Vim/vim73/gvim.exe\\""

Command line option parsing

webmaster@gewalli.se (Oskar Gewalli) — Fri, 07 Oct 2011 05:58:00 +0000

I’ve updated the dotnet-console to enable help. It’s not finished, but it’s looking better. Still needs a lot of refactoring though. The tests document the more stable fluent interface. The stuff behind the fluent interface is more prone to change. It’s an interesting experience to code the tests against a higher level api that is relatively stable (as an interface) and have an underlying changing implementation.

Having at least 2 monitors?

webmaster@gewalli.se (Oskar Gewalli) — Sun, 02 Oct 2011 21:37:00 +0000

Why so many monitors? I used to have the same kind of layout on my home computer while at the university by using X11. You have virtual desktops (not to be confused with the software) or workspaces.

The same concept on mac os x differs significantly from how it works on linux.

Part of the problem might be technological: When I’ve tried out virtual desktops on windows, the programs become extremely unstable.

User friendly: The concept of virtual desktop is probably not very conceptually friendly for a person without a lot of experience with computers. Since power users are assumed to have enough money to buy enough big monitors.

Simple noise on the arduino

webmaster@gewalli.se (Oskar Gewalli) — Sun, 02 Oct 2011 19:04:00 +0000

Inspired by the minimalists I tried this out on the arduino:

int speakerPin = 11;  
  
void outp(byte value) {  
    analogWrite(speakerPin, value); delayMicroseconds(200);  
}  
  
void setup() {  
    pinMode(speakerPin, OUTPUT);  
  
  
    // Set up Timer 2 to do pulse width modulation on the speaker  
    // pin.  
  
    // Use internal clock (datasheet p.160)  
    ASSR &= ~(\_BV(EXCLK) | \_BV(AS2));  
  
    // Set fast PWM mode  (p.157)  
    TCCR2A |= \_BV(WGM21) | \_BV(WGM20);  
    TCCR2B &= ~\_BV(WGM22);  
  
    // Do non-inverting PWM on pin OC2A (p.155)  
    // On the Arduino this is pin 11.  
    TCCR2A = (TCCR2A | \_BV(COM2A1)) & ~\_BV(COM2A0);  
    TCCR2A &= ~(\_BV(COM2B1) | \_BV(COM2B0));  
  
    // No prescaler (p.158)  
    TCCR2B = (TCCR2B & ~(\_BV(CS12) | \_BV(CS11))) | \_BV(CS10);  
  
}  
  
void loop() {  
  int t;  
  for(t=0;;t++){ outp(t\*((t>>12|t>>8)&63&t>>4)); }  
}  

It does sound somewhat similar. So right now it’s just a hack in order to try it out.

Note:
The code above is corrected with a little help from Davey at forsken.

Implementing Dispose method in .net

webmaster@gewalli.se (Oskar Gewalli) — Sun, 25 Sep 2011 19:30:00 +0000

Dispose

In order to release unmanaged resources you need to use or implement the dispose method.

Unmanaged resource

So what is an unmanaged resource? It is simply resources that the CLR does not manage for you. For example: File streams, database connections, P/Invoked api:s.

How to implement Dispose?

Msdn gives a rather clear answer.

You need to implement the IDisposable interface.
If you have implemented a finalizer as well then you need to add a call to GC.SuppressFinalize inside the dispose method. This is to avoid that the finalize method is called for the object.

When is it unsafe to call both dispose and finalize?

One reason could be that you’re dealing with a P/Invoked api. Telling the unmanaged code to clean up the resources associated with a particular pointer twice can cause unintended consequences. Using closed handles is a security risks since windows recycles handles. Another reason is performance: A finalize is costly to perform.

Alternative to using IntPtr

In order to simplify the use of operating system handles you can use “Safe Handle”.

Windows 8

webmaster@gewalli.se (Oskar Gewalli) — Mon, 19 Sep 2011 20:26:00 +0000

I like the look of windows 8. For touch devices it will be sweet. What confounds me is that the start menu is replaced by the “Metro” application launcher. That sounds exceedingly annoying when on standard hardware. Where is the love for an ordinary PC?

The reason for the weird usability is that it is intended for touch enabled computers:

I’m confused about the talk about the name Windows 8. Is it supposed to be a general windows?

WinRT replaces win32

webmaster@gewalli.se (Oskar Gewalli) — Mon, 19 Sep 2011 20:21:00 +0000

Sounds interesting to hear about the new WinRT api:s for windows. Reading up on it WinRT are COM objects in contrast to the Win32 architecture. One of the exciting things about this is the promise of being able to use xaml in c++.

An overview of WinRT by Marius Bancila: Windows Runtime

It’s quite interesting to read the list on Msdn: Windows Runtime and the Windows API

Right now it’s probably to early to get a good idea of how everything will look and exactly what features will be accessible for Metro style apps.

Replacing the Win32 api:s is probably an overstatement. I would guess that for some things you might be forced to use older api:s.

Running Rake under cygwin

webmaster@gewalli.se (Oskar Gewalli) — Thu, 01 Sep 2011 13:14:00 +0000

I had some problems with running rake under cygwin. I get the following error while trying to execute rake:
C:\Ruby192\bin\ruby.exe: No such file or directory – /cygdrive/c/Ruby192/bin/rake (LoadError)

Could it be that the problem is that the first line in rake:

#!/usr/bin/env ruby

Will return C:\Ruby192\bin\ruby.exe

By being more explicit about what to run you can avoid the problem:
rake.bat

Writing code without tests

webmaster@gewalli.se (Oskar Gewalli) — Wed, 11 May 2011 14:58:00 +0000

Suppose that you write code without tests that compile. Suppose that those parts do not have any user interface or tests. Then what you’re doing is actually pushing development cost over to other parts of the project that make use of the code. This will probably mean that any cost/time estimates concerning dependent code needs to be adjusted in order to be able to pay for the additional cost.

Command line option parsing

webmaster@gewalli.se (Oskar Gewalli) — Sun, 24 Apr 2011 18:10:00 +0000

NDesk Options

I started writing an argument parser without knowing about: NDesk.Options Where you have the following kind of syntax:

string data = null;
   bool help   = false;
   int verbose = 0;

var p = new OptionSet () {
    { "file=",      v => data = v },
    { "v|verbose",  v => { ++verbose } },
    { "h|?|help",   v => help = v != null },
   };
   List extra = p.Parse (args);

FubuCore

Since this post, there is a new option: Command line in FubuCore.

Powershell

If you’re using powershell it just might be simpler to load your program as an assembly and invoke the classes directly. Looking at stackoverflow there is a command Add-Type. Otherwise you could use standard .net assembly load.

Isop

All of the solutions above are not really what I’m after. Thus I’ve written another argument parser loosely based on SimpleConsole : Isop I didn’t really think views in an console app was that useful. Instead I’ve tried to do some of the plumbing in order for you to write console applications more like this:

static void Main(string[] args)
{
   new Build()
      .Parameter("server",arg=>server=arg)
      .Recognize(typeof(CustomerController))
      .Parse(args)
      .Invoke(Console.Out);

}

There is some simple syntax to extend this. To Set culture, TypeConverter et.c.

So you would invoke the method “Add” on the class “CustomerController” with the arguments id and name:

CustomerConsoleApp.exe Customer Add --id 1234 --name Arne

Testing SQL database

webmaster@gewalli.se (Oskar Gewalli) — Sun, 17 Apr 2011 13:40:00 +0000

What strategies do I know of on how to test your code depending on a database?

Using TransactionScope so that you can roll back any changes done by tests.
Setting up a fresh database with the same schema.
- Using NHibernate SchemaExport(cfg).Execute to export an existing schema onto the new database.
- Run your migrations to set the database in the correct state.
- Use Code First, auto migrations or whatchamacallit call it. That is, let the ORM generate the tables and columns based upon your data objects. For instance Entity Framework Code First or Subsonic Auto Migrations.

What are the pros and cons of these approaches?

TransactionScope:
- Pros: Very easy to setup. If you have a local version of the database, it’s very easy to get started.
- Cons: Running everything under a transaction can be very costly. You need to tweak the configuration of the transaction coordinator. If you forget to use TransactionScope, there is a danger that you get rubbish data in your test database making your tests brittle.
Fresh database with the same schema:
- Pros: By having migrations that are well tested, it can be considerably easier to create new installations.
- Cons: Using a slightly different schema for your tests (for instance if you have sql scripts to do changes to the database and turn on Code First in your tests) can lead to unexpected behavior. If your logic depends on database specific functionality you might want to use migrations in order to have a fully working schema. It can also be a bit harder to setup. If you use a different database (for instance SQLite), database specific behavior may be an issue. Same vendor but different collation might also give you a headache.

Build/test time

webmaster@gewalli.se (Oskar Gewalli) — Sat, 16 Apr 2011 21:17:00 +0000

Keep solutions simple and think about your deliverables

Chad Myers at Los Techies tries to get a discussion about why it’s bad to have to many projects: Project anti-pattern: Many projects in a Visual Studio Solution File Some of the consideration is about build performance. He is referencing microsoft blog posts… I know the lure of getting to drink coffee while waiting for builds, but you’re not getting paid for coffee ;) There is an article on O’Reilly about the cost of long development cycles: Dead Time (…code, compile, wait, wait, wait, test, repeat)

This is also reflected in the documentation on team development: Msdn: Team Development with Visual Studio Team Foundation Server: Chapter 3 – Structuring Projects and Solutions in Source Control They recommend you to keep it simple:

Keep the following considerations in mind when designing a project and solution structure:

Each project generates an assembly at build time. Start by determining what assemblies you want to create and then use this to decide what projects you need. Use this to determine how to factor your codebase into projects.
Start with the simplest single solution structure. Only add complexity to your structure when it is really necessary.

…

From Brad Adams msdn blog: Perf trade off: lots of small assemblies or fewer bigger assemblies?? Puts it like this: fewer bigger assemblies are better

But he is talking about the performance of your applications. He mentions that they optimize for bigger assemblies. This is probably why visual studio is optimized this way as well.

Why writing unit tests and not only integration tests?

Integration tests are immensely useful. However, the amount of time you spend on running tests is part of the above mentioned coffee time. If you are writing unit tests when testing your C# code this means less time spent being bored waiting.

About why build time is important

Note that they are even factoring in the time it takes you to run the tests. Imagine the Dead Time if you need to test manually. Not to rant, but the Dead Time ratio would be even higher. It’s simple economics that by writing tests as part of your development cycle will make you more productive.

How often do you execute a build per day? Lets say that you do it every 30 minutes ca. If your build time is 1 minute. That gives you a 31 minute cycle. Divide 8 hours by this cycle and you get: 15.5. If the build time is 5 minutes that gives you: 13.7. That means you will be 12% less effective per day.

The above mentioned number is actually misleading. An experienced developer working with well known tools may be able to work in longer cycles thus avoiding a lot of the Dead Time. However, when working in areas you are relatively inexperienced in (only worked with for a few years) you will benefit from the possibility to do explorative coding if your build/test cycle is short. Michael Feathers touches on this issue in his post about the Urge to REPL, the Urge to test. REPL is simply read-eval-print-loop. It’s why you can be very productive when you use scripting languages. You can do explorative coding and get instant feedback without waiting for builds.

The numbers also suggests that monitoring your build time, test time can lead to greater productivity. Maybe not running all of the integration tests but letting a continuous integration server let you know if any of them fails.

Another important factor is: Instant gratification. The joy of seeing the result of your action in a short amount of time.

Also you might want to read

Partitioning Your Code Base Through .NET Assemblies and Visual Studio Projects

I Don’t Have Time for Unit Tests

When to use mocks?

webmaster@gewalli.se (Oskar Gewalli) — Sat, 16 Apr 2011 07:27:00 +0000

You use mocks when you want to test how the code interacts with the mocked objekts.

If you care mostly about the end result, the final state, then it’s easier and less brittle to use fakes.

Migrations

webmaster@gewalli.se (Oskar Gewalli) — Fri, 15 Apr 2011 19:49:00 +0000

Listening to herding code on fluent migrator is quite interesting.

You could read Getting started with fluent migrator. If you feel the need to contribute there is fluent migrator on github.

HOWTO write something FAST that works!

webmaster@gewalli.se (Oskar Gewalli) — Sun, 03 Apr 2011 13:18:00 +0000

Focus

Make sure that you’re always doing the right thing (Erik Arnwald). Try to understand as much as possible of the domain you’re working on.
Trim furiously in the specifications and in the code. Make sure to identify the key benefit for the customer and focus solely on implementing this part. Additional wishes and demands tends to change rapidly and might not be necessary at all. The most common reason to break of deadlines and malfunctions is too complex specifications which are changed after the project has been started. (Carsten Hersom)

Avoid coding

Sometimes you do not need to any custom code but can use standard components coupled with some education of the customer. For instance a standard CMS module. (Carsten Hersom)
Make sure to use stable and well tested third party tools and libraries for your core effort. For instance. The guys maintaining NHibernate have extensive test coverage of the library. Avoid any tool that is not either really stable and old (and is used by many) or has a good test coverage.
The same criteria as for other tools apply to JavaScript frameworks and libraries.

Avoid breaking your own things

Make sure that everybody uses database versioning tool. Preferably some form of migrations. It’s trivially easy to get into a inconsistent database state otherwise.
Make sure to have test coverage of any part of the code you’re unsure of how it works or any brittle parts. Things that are brittle is for instance the use of SqlCommand against stored procedures (SP). As the SP’s are changed code may stop to work without you knowing it. Make sure to have tests of the connection between C# and the sql database. Make sure that your tests are as terse as possible. Don’t be afraid to throw old tests away that are wrong and not needed. You can read about tests in the post: why automated tests.
Be really careful of hiding exceptions (catch the right exception). If something goes wrong, it should explode. It’s really hard to find errors if you have written something like: try{ ... } catch(Exception){}
Use static analysis to your benefit. Turn off most of the crap (for instance fxcop style rules). Make sure that fxcop only reports relevant information. If unsure, avoid using this tool. Javascript lint is your annoying Besserwisser friend. Change the rules according to your need. You can download an integrated version for visual studio. Make sure to change what it reports (and that it does not complain about third party js libraries).
In order to avoid the time spent running integration tests, you might want setup a CI environment like team city. Make sure that you instill the value of tests in the developers you are working with.

On writing tests

You may ask: Why writing tests if time is short? The reason for this is that the time spent manually debugging and testing can be considerable. I’ve read estimates of the time spent in software development is mostly fixing bugs: Thus debugging and manually testing.

Some people can write a working program really fast without tests and structure and with mostly duct tape and spit to keep it together. These people are few. Of the programmers I’ve met only a handful have this skill. For most of us it’s not quick and dirty, it’s just dirty. As one of my former coworkers (Mads) told me: Writing quick and dirty code is like taking a loan. There is always an interest that you need to pay. The longer you wait before refactoring the higher the cost.

Feedback

Start by making a small part of your program that can be shown to people for feedback.
Release as soon as possible so that others (customers and domain experts) can give you feedback. It’s always better to have complete workflows in the gui (even if some of the functionality is faked). It helps the users and customers to visualize and provide better feedback. (Erik Arnwald)
In order to get people to contribute feedback on the current site/app you might want to consider automated deployments. This can help you when you need to deploy quickly for some reason (bugs). It can also help people unfamiliar with all the intricacies of a projects deployment to help with this activity.

Mentality

Don’t be afraid. If the functionality is to complex, try to understand the problem the functionality is supposed to solve. It might be that you can solve it with less complexity. If you’re afraid of stuff breaking, write tests.
Avoid having people around that stress you or your coworkers. It might be better to have stressed managers or customers interact with only one person from the team. Productivity slows to a crawl if people get to stressed.
You need to talk and listen. The first thing that happens to me and others are that you stop listening when you’re stressed.
If you are in many projects that stress you out, you burn out or make an ass of yourself (burn yourself in the eyes of others).

Why automated tests?

webmaster@gewalli.se (Oskar Gewalli) — Thu, 31 Mar 2011 21:14:00 +0000

Work is more fun when the things you create mostly works.
When fixing bugs, the probability of introducing new bugs is reduced.
Specifications of program behavior expressed as tests are easier to maintain than tutorial code or a word document specification.
In order to be able to write unit tests easier you are forced to simplify your classes.
Switching underlying technology or infrastructure. For example: If you have integration tests on your database repositories or data access layers it is easier to switch the underlying implementation. You have a way of verifying that important behavior is intact.
If you are writing code that:
- Requires the use of the debugger frequently.
- After you have written it, you fear to change.
Do you really consider yourself a professional developer?

Some of it is in relation to having to work late at night and during weekends. Tests can help you with your definition of done, thus alleviating the need for late night coding sessions.

Effective coding means to use the tools available when they are appropriate.

Sometimes it’s about doing something simple:

Is the bug to complex to find using tests? The flow of the application is wrong…

Maybe you could find it easier if you use a logger to trace the behavior?
You have written a test that recreates the problem, but you cant figure out why it doesn’t work.
Maybe putting breakpoints and debugging your test will help you find what’s wrong.

If you’re a beginner. How do you start writing tests?

If you’re writing C# then you want to read “the art of Unit Testing” by Roy Osherove.
If you’re unsure about how to write tests. Try using test lint.
Search posts done by Martin Fowler, Kent Beck, Robert C. Martin (Uncle Bob), Michael Feathers .
How do you do when you test your code manually?
1. You set up the state of the program relevant before the method/procedure to test. In NUnit it’s called SetUp. You use this attribute to mark the method that sets up the state.
2. You fill in a form and click on a button, browse to a page with the right parameters in the url, et.c. When testing this is the actual test. You mark a method that does this by the Test attribute.
3. You look at the result from the action. In NUnit you use the static helper class Assert that contains methods to help you write the expected values.
4. After you’re finished testing, you may need to clean up. In NUnit this is accomplished by marking the a method with the attribute CleanUp.

If you are structured in your testing of your app you may know some of the domain of automatic testing. You can read about this in the above book by Roy Osherove.

Also, writing tests is not a reason not to test what you’ve written manually. It is a way to make sure that you don’t have to do as much manually testing as before.

Role Playing games

webmaster@gewalli.se (Oskar Gewalli) — Thu, 31 Mar 2011 19:47:00 +0000

In real life you choose what parts to level, what to learn.

As a developer do you choose to level up or are you stuck fighting the same kind of problems without gaining XP?

As a developer do you choose to spend your experience:

Learning to talk to customers
Learning to work with others
Learn patterns, how to write prose
Learn new technologies
Learn how to write tests
Learn how to automate tasks (build, installation, et.c.)

This is just a short list. What I mean is that you have a possibility to choose certain skills. Probably you will ignore some of them. You will be mistaken if you think that the rest of the skills does not matter. Knowing what customers talk about matter! Being able to write correct software matter (that is, being able to write tests)! If you have chosen not to learn one of the skills, know that you will need other persons to lean on. You need to treat these people with respect. They are not dumb just because they are focused on parts you have not chosen to focus on.

log4net

webmaster@gewalli.se (Oskar Gewalli) — Wed, 23 Mar 2011 21:08:00 +0000

For parsing log4j files there are some alternatives:

Chainsaw (java gui). If the links dead, note that it should be part of log4j.
Similar to chainsaw but in .net: logviewer.
Log2Console looks pretty sweet. It has some positive comments.

On the logviewer page you find a sample configuration for using “log4net.Layout.XmlLayoutSchemaLog4j”. Note that the log4net xmllayout should be parse:able by logviewer as well.

Both NLog and Log4Net can log to the log4j layout.

Logging for Commerce Server 2009

webmaster@gewalli.se (Oskar Gewalli) — Tue, 08 Feb 2011 20:23:00 +0000

You might want to start off by reading the documentation from Microsoft. The short version of this text is that in the new CS2009 there is tracing and it is done through system.diagnostics. I don’t know the entire story, but it looks like the exception handling is piped through the trace.

So in order to configure the logging for commerce server you will need to read up on the system.diagnostics tracing.You will need to add a source ”Microsoft.Commerce.Exceptions”. There you can choose your TraceListener and specify the information you need by reading up on TraceOptions.

The default trace configuration can be found in SystemDiagnostics.xml in the Extensibility kit. You should be aware of that the default configuration logs to the Event handler. This is not always appreciated by your customers since they may want to have Medium Trust for the site.

Note that this means that you can also write your own log-handler for these tracing events by inheriting from TraceListener.

Whats wrong with Commerce Server 2009

webmaster@gewalli.se (Oskar Gewalli) — Wed, 26 Jan 2011 18:00:00 +0000

Whats wrong with Commerce Server 2009 from a developer perspective?

Configuration: There is no single source of object mapping configuration. Multiple places to insert properties.
FaultException does not include the original exception.
The CS2009 api’s look very dynamic… It promises a lot more than it can deliver since it is based upon CS2007 API and thus quite limited.
No transactions for some of the entities. For instance profiles and catalog.
Integration with SharePoint by putting the configuration in SharePoint web.config. That’s not the way to go.
Granular permissions for all the web services, for all the databases. There is a lot of work just setting up CS.
Pipelines….

How do you improve the experience?

Configuration: Just try to write integration tests to catch any errors due to configuration.
Problem with FaultException is fixed in CS2009 r2 CTP … Until an actual release, you might need to write your own logging for operation sequences (theres an extension point for this, just look at Microsoft.Commerce.Common.TextLoggingExceptionHandler and the configuration). I’ve not tried this, so I have had to look in the console output or Event log.
You could, if you had the time, write a different implementation of the operation sequence that does the queries for the basket or any other CS entity. I don’t know what level of the API the pipelines depend on. Probably easier to just ignore this issue and write some code that searches the CS database.
This is a serious issue. This means that you will need to clean up manually after any integration test. Just keep your integration tests small and write fake implementations of the CS entities repositories so that you can write unit tests.
If you’re not using default configuration and don’t need any serious modification of the web parts distributed with CS2009 then try to as early as possible, put CS within it’s own web site and implement your web service wrapper around the methods using CS. Could be a bit tricky, but might save you a lot of work. It’s simple math, one big product configuration inside another big product with a lot of configuration equals a lot of work if you’re unlucky. I’ve heard rumors that Microsoft is implementing a service oriented version of CS (but until then, you’ll have to implement it yourself).
Permissions: RTFM! To install CS you need to make sure that everything is setup correct.
Pipelines: If possible try to avoid using them.

Refactoring multiplicity, synchronous versus asynchonous

webmaster@gewalli.se (Oskar Gewalli) — Sat, 18 Dec 2010 12:00:00 +0000

Sometimes change hurts.

You know what I’m talking about if you need to change the multiplicity of a relationship (let’s assume you talk about an ordinary sql database). For instance: From 1 to 1 To 1 to many For instance, say it’s an address. The assumptions made when there is only one address permeates throughout the applications. For instance a person must have bills sent to a different address (his/her accountant or something similar). However, when contacted, any mail needs to be sent to the home address. The code changes alone could be many. If there are a lot of reports based on the data (accessing the information without any layer) then these could be based on assumptions out of sync with reality. If you’re unlucky that might mean that you need to manually go through all the reports in the system and look for discrepancies. If you’re lucky the report will simply fail. If unlucky, it will show a random address.

Synchronous versus asynchronous execution and parallel execution jQuery has popularized asynchronous programming. Reactive extensions will perhaps simplify your tasks. Parallel LINQ (PLINQ) and the Task Parallel Library (TPL) might even make the tasks even simpler. But what if your program assumes execution in a consecutive manner? That the parts that you need to make parallel or asynchronous are buried deep within layers of layers … It might be that your program design needs to be restructured.

Why git or mercurial?

webmaster@gewalli.se (Oskar Gewalli) — Fri, 10 Dec 2010 22:21:00 +0000

I’ve played around with branching and merging in git. It looks like a useful tool.

In the current project we are maintaining two different branches right now. I can see that it would be useful to be able to use the functionality in git/mercurial to help me merge and cherry pick.

HERDING CODE 96: ERIC SINK ON VERACITY AND DVCS

Hanselminutes Podcast 231 - Eric Sink on Distributed Version Control Systems

What's wrong with SharePoint

webmaster@gewalli.se (Oskar Gewalli) — Fri, 26 Nov 2010 23:17:00 +0000

Whats wrong with SharePoint from a developer point of view?

Packages
- It’s somewhat unreliable in it’s deployment of files.
- Takes a lot of time to install/uninstall a package.
- Not a modern approach to package management. Its basically a cab file with files and xml describing it.
Configuration
- How do you test the deployment of configuration?
- How does it work exactly? You have to observe the result … It only works when you use it the right way and you don’t get much in error messages.
- Deploying the right way is frikkin hard. But, if you expect the tools for sp to work without crashing your site, you better do it the right way.
From line of code to result. Could this be described as lead time of changing a line of code to viewing the results of it in the interface?

10 min or more if you’re unlucky. This means that the roundtrip is not comparable to a clean site in asp.net.
Not even a mediocre web platform. Its written for intranet. This shows.
Running code in SharePoint context can be up to 10 times slower for the same dll.
Security. Some of the problems with SharePoint are related to security. It adds to the complexity.
You need to mark your controls as SafeControls.
Can get you thinking that GAC deployment is a good idea. Also you need to be aware of asp.net security configuration and what you may do. If you put things in GAC without very good reason do not complain if visual studio crashes, your projects use obsolete versions of your code, your build crashes.
Identity Impersonate…
Huge. There looks like there at least 3 ways of doing things, and they are all for slightly different scenarios. This is an issue. You have to learn a lot of SharePoint concept in order to get the job done. You have to do extra work to introduce for instance an asmx. This is not helped by asp.net being a leaky abstraction for producing web sites/apps. It can work well if you accept the behavior and limitations of of asp.net. Problem is that it’s not often that people are satisfied with this. Thus it may help to move to asp.net mvc for a simpler more web centric view.
If you’re supposed to develop web, it’s not a good experience.

Compare this to a regular asp.net web app/ site.

How do you improve the experience?

Conventional wisdom apply:

Testing your logic using automatic tests is valuable. Even more so since any changes take a lot of time to enter into SharePoint.
If you can simplify by faking SharePoint functionality
- Writing web in regular web and move it using merge or generation
- Write simpler versions of SharePoint classes.

Use fewer dlls. Use ilmerge or consider fewer projects in your solution. Good idea anyway since its good practice to have few projects in visual studio. In .net its best practice to have few dll:s. In visual studio it’s better to have few projects in your solution. You can solve this by having good build scripts and small sub solutions (may not be a good idea if you do not have a lot of code).
If possible avoid putting things in the GAC. It can be useful, but not for everything. You can put dll:s in the bin folder of SharePoint.
Automate as much as possible.
Write build scripts.
There is a sample macro for attaching to a process.
If there are no logs or anything to help you with what’s going wrong: You might need to use advanced tools like Process Monitor.

Some other helpful tips may be:

Wspbuilder can be a good starting point. It can help you by automatically registering safe controls. Ms commerce extendability project can show you how to do it without the tool. Some of the uses for this tool might become obsolete with the introduction of Visual Studio 2010 and SharePoint 2010 (might not save your bacon though). I’ve not tried this, only read it on blogs. It might help you to understand what the tools generate.
Be pragmatic when it comes to configuration. Don’t hesitate to use a merge program when your configuration feature fails. Unless of course your client will use the wsp on a SharePoint farm.
Put your code behind wcf services so that the configuration and context is outside of SharePoint. Less configuration helps. Also the separation of your code and the executing context in SharePoint help you configure security easier.
Don’t be stressed if it takes time. Read up on the things your working with (try to ignore the rest).

XML build files are obsolete

webmaster@gewalli.se (Oskar Gewalli) — Sat, 20 Nov 2010 16:59:00 +0000

Why using a build system like Ruby rake or some other next generation tool (psake looks promising) to build your .net application or library?

A) Builds are not data. Xml is for data. It may be that the files or configurations are stored as xml, but does your build program need to be in xlm? B) Custom tasks are easy to write. Take a look at albacore for instance. C) It is easy to manage nontrivial tasks. D) It’s readable. Write your build in rake or something similar. Write it in msbuild. You will see that as usual there is a cost to writing it in xml: It adds cruft and that angle-bracket tax.

Or you might like the xml files for some reason: Jonathan Pryor: Defending XML-based Build Systems He is arguing for the use of xml-basked build files. One reason is that xml is easily parsed and can be used in custom tools.

Personally I feel that coding should be done mostly by writing code. Sure, there might be nifty code generation tools that can be used to simplify some tasks, but at the end of the day you still need to get your hands dirty and make sure that it does what it’s supposed to do.

Simple object oriented JavaScript

webmaster@gewalli.se (Oskar Gewalli) — Fri, 30 Jul 2010 07:48:00 +0000

The jQuery approach is to avoid object orientation. This is an excellent way of avoiding bloat for small JavaScript hacks. If you’re starting to build JavaScript heavy applications then you might need to start organizing your code. The first thing you should learn is how to write jQuery plugins. Later on you will need to learn some simple object orientation. Here’s how:

function TigerType(name){
    var that = this; this.name = name; this.sound = "growl!";
    this.roar = function(){ return that.sound; };
}
var tigerInstance = new TigerType();
alert(tigerInstance.roar());

Note that I’m declaring a variable “that”. If you work with jQuery you know that this will be set to something else. By using a scoped variable you will avoid this problem. So what about inheritance? The simple solution is to use call or apply.

function TigerWithTeethType(name){
    var that = this;
    TigerType.call(this,name);
    this.bite = function(){ /*some code to bite the programmer ;)*/ };
}

Since TigerType is a function, call and apply can be used to execute it with the “this” of TigerWithTeethType. This way has the advantage of being simple and jQuery safe. You might also want to look into:

Try to avoid object orientation when writing JavaScript. If you’re doing small things then there is no need for the added complexity of object orientation (in JavaScript).

Different breed of programmers

webmaster@gewalli.se (Oskar Gewalli) — Mon, 19 Jul 2010 19:46:00 +0000

After talking to a few of my colleges I’m thinking of the different kind of programmers needed for different projects, for different aspects of projects.

Duct tape programmer: http://www.joelonsoftware.com/items/2009/09/23.html Writing code that ships. Does not necessarily need to look clean. Will probably not write any unit-tests. No fancy abstractions, no patterns or anything.

Maintenance programmer: http://www.codinghorror.com/blog/2006/06/the-noble-art-of-maintenance-programming.html Have had every sort of wrench imaginable thrown right in the face. This guy will be very skeptic of dirty code. It’s about pragmatic programming and excelling at making sure that old stuff works.

The thing is: Different type of programmers are good for different things in a project. A maintenance programmer (or a programmer in that mode) will not necessarily be any good at writing new applications FAST (like really, really fast). Maybe the sales force needs something done within a week or a few days. Then you’re looking for a duct tape programmer. But don’t kid yourself. Finding people good at this kind of skills is hard. Problem is, you might loose time by pressuring the other programmers into writing code fast. People might think that they are duct tape programmers. But seriously, few of us are.

Duct tape programming, the term that comes to mind is Quick & Dirty As a coworker said: - When you do it quick and dirty you have actually taken a loan. If you take additional loans to do small fixes, sooner or later any change will take horrendous time.

Where do I stand in this mix? Maybe I’m a mix of a few of these. Frankly I’m not very good at duct tape programming. I try to mix maintenance and duct taping. Doing integration testing and unit testing in order to make sure that the server bits are done until the time that I start coding the GUI. Making sure that the GUI interacts with a decent API instead of doing the logic directly will make it easier if you have a lot of complicated server logic. The pro of this approach is that you do not need to be a pro at the tool you’re using. As the requirements change, you will still have something that works.

Dojo RTE editor (dijit Editor)

webmaster@gewalli.se (Oskar Gewalli) — Sun, 02 May 2010 09:43:00 +0000

Finished upgrading an old site to use dojo and codeignite. In order to simplify things I added the RTE editor in order to simplify the code. There is no need to add logic related to the content on the page so html-content is preferable.

excerpt from the content/editor.js:

dojo.require("dijit.Editor");
dojo.require("dijit.form.TextBox");
dojo.require("dijit._editor.plugins.FontChoice");
dojo.require("dijit._editor.plugins.LinkDialog");
dojo.addOnLoad(function(){
    dijit.byId('bodyeditor').setValue(dojo.byId('body').value);
    dojo.query('[name="submit"]').onclick( function(evt) {
         dojo.byId('body').value = dijit.byId('bodyeditor').getValue();
    });
});

and from the view editor.php:

$actiontitle

Titel:

Body: <? echo set_value('body',$body) ?>

Note the dojoType attribute of the editor div:

dojoType="dijit.Editor"
extraPlugins="[
    {
    name:'dijit._editor.plugins.FontChoice',
    command:'fontName',
    generic:true
    },{
    name:'dijit._editor.plugins.LinkDialog',
    command:'createLink'
    }]"
id="bodyeditor" required="true"

As you can see, I opted for the more declarative way.

There were some pitfalls. I could not use the editor directly for instance. I had to define a hidden element for the content and post that element. Also, the declarative statement looks kind of ugly since we have to add the full name of the editor plugins.

The first thing you notice of dojo is that the api isn’t as slick as jquery. That is expected since dojo spans more functionality. dojo shines however in one position: dojo.require. Like the import statement in java and python or the using statement in C#.

Playing around with mongo db in .net

webmaster@gewalli.se (Oskar Gewalli) — Mon, 05 Apr 2010 13:51:00 +0000

Ever since I heard of the NoRM driver for MongoDB I had to try my hands on it. You can find it here: NoRM , mongodb

The first thing is setting up the server. I dropped the binaries into

C:\\MongoDb\\

Next thing I created two batch files. To start the db: start.bat

C:\\MongoDb\\bin\\mongod.exe --dbpath C:\\MongoDb\\data --port 27000 --logpath C:\\MongoDb\\log\\mongodb.log --logappend

The other to access the mongo shell: shell.bat C:\MongoDb\bin\mongo.exe –port 27000

Next thing is creating a simple asp.net MVC app that uses MongoDB for storage.

My object of choice is Product:

public partial class Product
{
  public Product()
  {
    Id = ObjectId.NewObjectId();
  }
  [Column(IsPrimaryKey = true)]
  public virtual ObjectId Id
  {
    get {return _id;}
    set {_id = value;}
  }
...
}

The attribute is for the editor template. Next thing is the controller. Here is the helper for connecting to the database:

  public class ServerHelper
  {
    public static Mongo GetServer()
    {
      return Mongo.Create("mongodb://localhost:27000/MongoMvc");
    }

    public static MongoCollection Collection()
    {
      return GetServer().GetCollection("products");
    }
  }

Note that you don’t need to do a create database or collection.

Insert is straightforward. In order to list all “products” I used an empty Find on the collection. To update I did the following:

    [HttpPost,ActionName("Edit")]
    public ActionResult EditPost(ObjectId id)
    {
      var collection = ServerHelper.Collection();
      var product = collection.FindOne(new{ _id = id });
      UpdateModel(product);
      collection.UpdateOne(new { _id = id }, product);
      return RedirectToAction("Index");
    }

Looks interesting. I’ve not delved into any deeper parts than this yet. I could probably have done this without the additional configuration.

Debugging JavaScript in IE8

webmaster@gewalli.se (Oskar Gewalli) — Sat, 18 Jul 2009 19:55:00 +0000

How do you debug JavaScript in IE? Before IE 8 you could use fiddler and the visual studio debugger.

There was a plugin for IE so that you could get a developer toolbar with some extended script debugging. As of IE8 a toolbar is built in and can be reached by pressing F12.

Debugging JavaScript in Firefox

webmaster@gewalli.se (Oskar Gewalli) — Sat, 18 Jul 2009 18:17:00 +0000

How do you debug code in Firefox? The answer is that you use one of two eminent plugins:

web api's

webmaster@gewalli.se (Oskar Gewalli) — Sat, 18 Jul 2009 18:00:00 +0000

Local procedure calls

Given the scripting nature of the web non-obfuscated JavaScript code presents itself to the technical user almost as a public API. You can try it out yourself by writing:

for (x in document){console.log(x);}

in your browser JavaScript console. For more persistent changes to a sites behavior tools like Greasemonkey can be used. Last.fm and Spotify have done some integration by releasing a Greasemonkey script for last.fm.

Remote procedure calls

By using fiddler or Firefox Firebug you can watch the client/server interaction (ajax calls made by a modern web app). The stateless nature of the web and ajax calls promotes a simple design of this “API”. This means that programming/scripting against your site can be easily done by, hopefully authorized, users.

Conclusion

Modern web applications have a natural interface for that might yield a plugin-like api unintended by the applications developers. If you’re developing/maintaining a web app, try searching for it on userscripts. If your app is used by a wide group of people, then you might be surprised with what you find.

Programming JavaScript

webmaster@gewalli.se (Oskar Gewalli) — Sat, 18 Jul 2009 16:40:00 +0000

In the first course in computer science we (students) received the following words of wisdom from the lecturer:

Above anything avoid JavaScript.

Pure JavaScript

A few years later I started working as a programmer (late 2005). Some small that we could things arose impossible without the use of JavaScript. One of my coworkers showed me how to add some bits of extra functionality that the users needed (easier user interface). Later on there was a need for drag and drop in the browser. I implemented this by using code from another coworker. I updated an editor for presenting study programs. I feel that that experience is a failure. It took me at least 2-3 weeks to get something that wasn’t decent enough. I tried to write good code, but failed. Both the server-side and client-side code is a mess. Fortunately this code is rendered obsolete by the fact that the system I worked on contains more information and therefore can generate such presentations without any manual editing. What I am trying to say: That coding experience sucked.

JavaScript library: jQuery

Fast forward to 2009. I’ve started working in a team developing what can best be described as an advanced editor (specialized for one purpose) on the web. Enter jQuery: Rich text editing, ajax, items that can be saved without the whole form being submitted, et.c.. The weirdness of developing JavaScript almost gone:

Doing manipulations on the HTML using jQuery yields more predictable results.
Navigating the DOM via jQuery works the same way across different browsers.
Events encapsulated by jQuery work the same in different browsers.
The terse syntax of jQuery reduces the amount of code.

Because of the above the quality of the code increases.

Lint

Since there is no compilation of JavaScript there is a need to use some form of lint on the code to catch syntax errors, legal JavaScript but bad syntax et.c.. There are two JavaScript lint implementations that I know of: JavaScript lint and jslint. I use both. Jslint is a bit harder to set up in visual studio, but is stricter and finds more errors (out of the box). I might not have read enough of the documentation for JavaScript lint in order to use it efficiently. With the use of lint hard to find bugs and errors can be found in a matter of minutes instead of days, makes these tools invaluable.

jquery

webmaster@gewalli.se (Oskar Gewalli) — Wed, 08 Apr 2009 22:17:00 +0000

I’ve come to admire the simplicity and style of jquery. There are some cons to it’s approach of using context (as can be seen as this in the for instance “each”):

jQuery("#table tr").each(function(){
    jQuery(this).DoStuffPlugin();
});

This wreaks havoc with some object oriented programming techniques that uses this for the object reference:

function ClassSmurf(){
    this.name = "smurfette";
    this.SmurfAround = function(){ alert("Smurf: "+this.name); }
};

This is a simple example broken by jquery’s use of context (that jQuery binds this). There is a simple work around:

function ClassSmurf(){
    this.name = "smurfette";
    var that = this;
    this.SmurfAround = function(){ alert("Smurf: "+that.name); }
};

Note that we capture the object instance as a private variable. We do get some additional boilerplate in order to have jQuery safe “classes”. Don’t take me wrong. I love that we have jquery.

Modelbinding a complex entity framework/Linq to Sql model to asp.net mvc

webmaster@gewalli.se (Oskar Gewalli) — Sat, 14 Mar 2009 05:41:00 +0000

For a couple of days now I’ve been trying to “ModelBind” a more complex model (based on the Entity framework) to a controller action. The problems I’ve been having are mostly related to the amount of code for this task. I’m therefore thinking of ways to reduce the complexity. The main problem is that it’s not directly possible to bind to ordinary EF/LS-graphs.

Separate object graph. One solution could be to have a separate object graph for use in with the view. That seems more than wasteful since we want to edit everything on this page and thus end up having to maintain extra objects when we have the EF/LS ones.
A revised binding to the EF/LS objects. If my model-binder treated the EF/LS objects in the correct manner then my problems would be solved. Problem is though that I need connected objects in the binding. That implies a UpdateModel pattern with the object context visible from the controller. Not a beautiful solution though.

I’ve written a small asp.net mvc application with a simple (complex model). I’ve a main entity Smurf. The Smurf entity has several SmurfChild records related to it:

The model binding I’m trying to do is the following:

public ActionResult EditAll_CustomBinder(
        [ModelBinder(typeof(SmurfListBinder))]
        IList<Smurf> smurfs
        )

I’m editing a list of smurfs and the related smurf children in the view.

Assert Fail

Hetzner mentioned in a few places

Prompt injection is the new SQL injection only easier

The world and US cloud providers

Evaluating different programming languages for use with LLMs

Experiences 2025

Convergence

Keywords

SWE-bench Multilingual

Token efficiency

We await more benchmarks

Devaluation of work

The point of source code

Productivity

Disruption

Conclusion

Slop bulkheads

Living with a machine ghost

Further reading

Monorepo conversion continued

Git Subtree

Subtree the good parts

Subtree the bad parts

Enter git-filter-repo

filter-repo good parts

filter-repo the bad parts

Conclusion

Clean Architecture Comparison continued

Background

Overview

Libraries in use

Conclusion

Programmers in the age of Copilot or GPT

Introduction

Terms

Tools that have had big impact on programming in recent history

Unit tests

The internet

Code completion

Stack Overflow

GitHub

Copilot

Alternatives

Where do we end up from here

Clean Architecture Comparison

High level overview

Samples

Comparison

Conclusion

Patch net7 dev container for net8

What does SOLID mean

Sources

Principles

Single responsibility principle

Open-Closed principle

Liskov substitution principle

Interface segregation principle

Dependency inversion principle

Conclusion

Data Specifications in Java and .net

Background

Sample based on Wikipedia

Higher level abstractions

Modules

Conclusion

Running minikube on M1 and K8S tutorial

Let the Yac Shaving begin

Running minikube in docker

Using qemu and building your own docker images

Side-note

Conclusion

Sample monorepo conversion

Project with many repositories

Time to merge

Git magic

Builds builds and builds

Conclusion

Running SQL Server on Docker with an M1/ARM CPU

Conclusion

Architecture and functional programming