3 Comments How code generation breaks YAGNI - 01/19/09

My post-title was getting too long, so I’ll just finish it here:

… and what you should do about it!

Actually, this is a bit of a follow up to post to my post about design principles. I felt I had something more to say about YAGNI in combination with software factories. Not very long ago, I talked about a software factory I layed my hands on.
Now, I’ve had the chance to mess around with it a little more.

Imagine starting a new project with a software factory. It generates a lot of code. Some of it usefull, some of it not very.
I’m asking two questions:
1) What code is actually usefull? => It depends on the case
2) What code is not usefull at all? => It depends on the case

According to YAGNI, we shouldn’t be writing code that we’re not going to use. But in this case, a lot of code we’re not going to use, has already been generated by the software factory. So the software factory breaks the YAGNI principle.
I don’t find it abnormal that the code generated by a SF breaks the YAGNI principle, but that doesn’t mean it’s OK to keep the code breaking YAGNI when using a SF!

Why does a SF break the principle?

A software factory, solves a common problem. But as we all know, every problem domain, has exceptions. A software factory will only be able to solve that what’s -generally- the same over all cases, thus the solution offered by the SF, will not be entirely correct in all cases.
Since we’re not starting from scratch with the codebase, the developer can’t avoid breaking the principle, since it’s the SF that breaks it. But as I said, that doesn’t mean it’s ok. While in cases we don’t use a SF, it’s the developer’s responsibility not to break the principle, now it’s the developer’s responsibility to clean up the mess the SF made.

Time to refactor the software factory?

If the SF is generating code that breaks YAGNI, doesn’t that mean it’s time to refactor? Well, yes and no. The SF is nothing more than an existing codebase, just like any other one. So it evolves with time, and like in every project, it also needs it’s refactoring love once in a while. We all know refactoring is a good thing.
During your refactoring-time, you could take into account what code to add, but most importantly since I’m talking about YAGNI here, what code to remove.
But that’s not an easy decision, since that code is usefull in one project, but not in the other one. The only code you can delete without thinking about it twice, is the code that’s just -never- used.
The code that deserves some serious brainstorming, headaches and analysis, is the code that is used in less than 80% of the cases.

The SF guys might be thinking: 80%??? Then you’re taking away 20% of value? And I’ll say, no, I’m not at all. I’m taking away the code, that is an added value for 20% of your projects, but that is an added misvalue (i had to give the kid a name) for the other 80% of the projects! I must say the developers that created this SF, thought about YAGNI carefully. If you only have about 20% that is YAGNI-code in code generated by a SF, I think we can say we’ve got ourselves a powerfull thingie here.

But still, you’ve got that overhead. That 20% of other code that’s only used in special cases.
You can’t delete it, still it bothers the hell out of you. And more important, it breaks the YAGNI principle! It makes your code base too big for nothing, and sometimes very confusing to work with.

Things I think should not be generated in the SF

- method overrides that only call your base implemenation: override them when you need them, typing “identifier override methodName” is not that hard, is it?
- tests that aren’t testing anything but prepare your mocks and variables: you could still have this partly handled by the SF, by creating a recipe for them

I’m sure I can add things, but these are the most obvious ones I’ve seen until now.

So what do we do about it?

Well, in my suggestion, I would just leave it in there… But not until the end of time!
You have to pay off your technical debt, and in this case it means you have to remove all YAGNI code! Plan some time to do this after you’ve finished a user story. If this seems to be too often, and you’re not removing hardly any code, you could choose to do this after each iteration. But don’t prolong it more than that, or you’ll end up not doing it at all, since it will take too much time to do it all in one go.

You’ll think, but what if I need that code in another user story, or an upcoming iteration? That’s where you’re breaking the principle. No excuses. After you’ve finished a user story or an interation, and you haven’t used the code, the chance you will be using it all, decreases a lot.
If you need it anyway in the end, just add it manually, it won’t kill you, I promise.

Important note!

If for some reason you need to regenerate your code (executing DSL’s for example), you should review all generated code for unused code again! And not after the iteration. I think in this case, it’s something you should do instantly. Do it at the same moment you regenerate your code. Consider it a part of regenerating your code. If you notice you’re regenerating a lot of code in different places while your change is small, then I would advice to take a look at your SF. It should offer you a way to regenerate parts of code, without affecting other ones.

How to identify YAGNI code?

Resharper already makes it easy to see YAGNI code (it will look gray) such as:
- unused overrides
- unused private methods
- unused parameters
- redundant code

But that won’t get you all the way through the cleanup. ReSharper only detects these redundancies locally (I’m hearing in a next version this option will be available solution wide), and it also gets harder with unused types. There are other tools that help you out with those problems.

FxCop is a great tool offered by Microsoft to perform static code analysis on compiled assemblies. On the other hand, -and I personally think this is the most powerfull code analysis tool on the market-, we’ve got NDepend. It analyzes your code, and let’s you examine dependencies, complexity, and  a bunch of other stuff using code metrics, graphs, other visual representations and -last but certainly not least-, CQL queries. NDepend already executes some CQL queries during the analysis, but afterwards you can even execute more specific CQL queries to analyze your codebase.

In our case, we’re looking for unused code. This CQL query is included in the analysis, so it’s veeeery easy! Just take a look at the results, and refactor them away.

For your information, here are the basic CQL queries NDepend uses for detection of dead code:

WARN IF Count > 0 IN SELECT TOP 10 TYPES WHERE
TypeCa == 0 AND     // Ca=0 -> No Afferent Coupling -> The type is not used in the context of this application.
!IsPublic AND       // Public types might be used by client applications of your assemblies.
!NameIs "Program"   // Generally, types named Program contain a Main() entry-point method and this condition avoid to consider such type as unused code.

WARN IF Count > 0 IN SELECT TOP 10 FIELDS WHERE
FieldCa == 0 AND  // Ca=0 -> No Afferent Coupling -> The field is not used in the context of this application.
!IsPublic AND     // Although not recommended, public fields might be used by client applications of your assemblies.
!IsLiteral AND    // The IL code never explicitely uses literal fields.
!IsEnumValue AND  // The IL code never explicitely uses enumeration value.
!NameIs "value__" // Field named 'value__' are relative to enumerations and the IL code never explicitely uses them.

WARN IF Count > 0 IN SELECT TOP 10 METHODS WHERE
MethodCa == 0 AND            // Ca=0 -> No Afferent Coupling -> The method is not used in the context of this application.
!IsPublic AND                // Public methods might be used by client applications of your assemblies.
!IsEntryPoint AND            // Main() method is not used by-design.
!IsExplicitInterfaceImpl AND // The IL code never explicitely calls explicit interface methods implementation.
!IsClassConstructor AND      // The IL code never explicitely calls class constructors.
!IsFinalizer                 // The IL code never explicitely calls finalizers.

If you haven’t looked at NDepend yet, I strongly advice you to do so!

Then, you’ll have a non-YAGNI codebase again.

12 Comments Downsides to software factories - 01/7/09

A while ago, I was quite excited about software factories. I had even joined a small group and we were going to explore GAT, DSL, and in general: software factories.

 

When do you need a software factory?

When you’re making the same solution structures over and over again, when you’re copying those nice and helpfull classes from the previous project, when you feel the problem your software is solving, is always generally the same… That’s when it’s time to consider building a software factory that easens the repetitive tasks in the development lifecycle.

 

A few definitions

The Guidance Automation Toolkit and the Guidance Automation Extensions are tools that help you to create optimized Visual Studio templates, alter the behavior of Visual Studio on specified project types, generating a solution structure containing several projects that again contain references, folder structures, and even code.

Wikipedia defines DSL or Domain Specific Language as follows:

The term domain-specific language (DSL) has become popular in recent years in software development to indicate a programming language or specification language dedicated to a particular problem domain, a particular problem representation technique, and/or a particular solution technique. The concept isn’t new—special-purpose programming languages and all kinds of modeling/specification languages have always existed, but the term has become more popular due to the rise of domain-specific modeling. Domain-specific languages are Fourth-generation programming languages (4GL).

You can create DSL’s to solve a common problem. Imagine a DSL that enables you to generate the needed configuration for your fav-ou-rite IoC within Visual Studio in a few clicks?

 

A nice software factory

Recently, I ran into a very nice software factory. It solves different problems:

  • - Create a solution structure, with a layered architecture
  • - Create testing projects for every project
  • - Generate stored procedures for CRUD-operations
  • - Generate the needed methods to call these procedures in the needed layers
  • - Generate tests that test the whole structure the factory generated
  • - Implements logging, tracing, transactions, … in the code

After all this, you can manually add stored procedures, run the DSL’s again, and that way update your code. The code that was generated, is placed in partial classes, so you can extend them. If you have to regenerate the code, your custom code is not affected.

The code is well structured, documented, and easy to read. But the design of the generated package, is completely data-driven. I’m not a fan of the data-driven approach, still, I know, in some cases, it can be the way to go. But I am conviced that in many cases, you can solve your problem the OO-way and you’d be better off!

 

Consequences…

The developers using this software factory, are capable developers. They know the whole structure of the solution, and do know what each layer’s responsibility is. They understand and master every piece of code this software factory produces, and that’s a very very very good thing :) .

However, in my opinion, the existance of this software factory, gradually can take away the open-minded behavior of a developer. I mean, that, when starting a new project, the first thing that comes into the mind of the developer, is using the SF, and that’s where you lost me. I know… the only reason you would invest in building a SF, is if you’re going to use it in several cases,… D’oh! But it shouldn’t be taken as a general rule for all projects. An SF that solves each and every problem domain does NOT exist and can’t be built either!

I agree that the software factory created a starting point for several successfull projects. I’m not saying it can’t make further projects succeed also… I’m only saying, that each and every project is different. What if it would be more convenient to build some new project using the OO or DDD-approach? That would mean more work. Yes, It would! So what? Is that a reason to fall back to the SF (and just choose the easy way)? No!

The usage of a SF should be considered for each and every project. There are some projects where the SF will offer you what you need to make it a successfull project. But don’t think the SF will make -any- project succeed.

 

Conclusion

I think it would be very interesting to build a software factory (it’s interesting looking at how it’s build, so don’t even get me starting about building one ;) ). A software factory should solve a common problem. I’ve actually always believed in them, but I never thought of the consequences in the mind of developers. I’m not saying every developer reacts like this (if it were so, I wouldn’t be writing this post in the first place…). Let’s state I’ve seen symptoms of what I’m describing here, and it’s unnecessary to state how wrong that is…

For the last time: Every project is different, some general concepts may repeat themselves, but still, the usage of a factory should be considered case-by-case.

|