~/ $ ssh server mysqldump mydb | mysql localdb
require 'db_subsetter' class Customer < ActiveRecord::Base end class Invoice < ActiveRecord::Base end class Charge < ActiveRecord::Base end class ChargeProductOption < ActiveRecord::Base end class Product < ActiveRecord::Base end class DemoSubsetFilter < DbSubsetter::Filter def customer_ids Customer.order(:id => :desc).limit(1000).pluck(:id) end def invoice_ids Invoice.where(:customer_id => customer_ids).pluck(:id) end def charge_ids Charge.where(:invoice_id => invoice_ids).pluck(:id) end def product_ids Product.where(:discontinued => false).pluck(:id) end def filter_customers(query) query.where(query[:id].in(customer_ids)) end def filter_invoices(query) query.where(query[:id].in(invoice_ids)) end def filter_charges(query) query.where(query[:id].in(charge_ids)) end def filter_charge_product_options(query) query.where(query[:id].in(charge_ids)).where(query[:option_for_product_id].in(product_ids)) end def ignore_tables %w( campaign_tracker_page_views campaign_tracker_visitors ) end end
# frozen_string_literal: true # A sample Gemfile source "https://rubygems.org" # gem "rails" gem "db_subsetter" #, :path => "~/code/db_subsetter" gem "mysql2" ss gem "pry"
Coping With Legacy Code
Sooner or later we all have to deal with legacy code. We’re thrust into a world from the ancient past, apparently before design patterns existed, where the code speaks a foreign language and our modern tools no longer exist.
How do you maintain, let alone improve, software in this dangerous world? Or worse, where do you begin when the right answer is to start porting the monolith?
Join us to discuss strategies and techniques for coping with legacy code. We’ll also discuss lessons learned while successfully porting overgrown legacy software with a small team. Some specific topics include:
- Porting a system, piece by piece
- Creating a sane development environment
- Getting a handle on change and source control
- Formalizing release and deployment processes
- Monitoring and containing what hasn’t yet been fixed
http://hackyourdepression.com/2015/02/why-you-cling-to-feel-bad-coping-mechanisms/ http://newinki.com/coping-strategies/ http://www.conceiveeasy.com/get-pregnant/coping-with-infertility/ http://oureverydaylife.com/coping-strategies-teens-16646.html http://www.toonpool.com/cartoons/Coping%20Strategies_183087
Intro - who am I
do we get right into it with “what is legacy code” then go into an overview, or do we do overview first? I like starting out with rube goldberg
- What is our problem?
- Cultural issues
- Technical issues
- What is legacy code? Code we don’t feel good about changing anymore
- All code becomes legacy code, so most code is legacy code
- Even that crap you wrote a month ago
- Why does legacy code go from annoying to a serious problem
- Lack of decent support/talent for platform
- Cultural problems at the dev shop - unwillingness to stop, correct, and prevent rot
- So there’s a lot of good information about refactoring and improving code (cite some books?)
- But we’re going to talk about the really gross problems
- projects in a dead/dying language that have to get ported
- projects where the architecture is so bad you can’t fix them
- oftentimes both!
I really struggled with the order to present these things, and I feel like some of this is repetitive and/or jumps around. But I really do think that a lot of the things we see in the worse legacy code problems all relate back to a few basic, fundamental problems. Sometimes the problems are technical, but often the problems really are deeply rooted in the culture. The problem, though, is that the everybody thinks they have technical problems and technical debt caused by management - nobody thinks the problem is culture in the technical part of the org.
Talk high-level about what we’ve been trying to kill at work - should I mention she she’s a pistol mess?
- We’re obviously mostly concerned here with the kind of systems that can’t just be rewritten quickly - big things
- These aren’t necessarily bad problems … having a good enough product to get out of hand means you must have had some success - management/stakeholders needs to understand that
- These can be really bad problems too, because they come down to cultural issues
- These are the lessons I learned, often too late, that often seem a little bit obvious but I wish I’d known and implemented sooner
- Types of problems you see
- Obviously, code is crap
- Dead or dying language.
- No specs
- The people you most need to help are often the least willing
- Accidental complexity - feature accumulation/compounding, but very little gets cut
- No clear SME to even know what isn’t important anymore (everybody uses only 20% of word, it’s just a different 20%)
- No SME - when you need requiremenst
- Everyone is an SME - when you make a decision
The people who do know the truth generally have a vested interest in not helping
Be brave, take stands, correct course.
- Avoid big-bang rewrite (these fail, often multiple times for one project). Requires too much time with no deliverables to keep people happy
- If your project is small enough you can rewrite/replace while customer isn’t looking, you don’t have big enough problems to be allowed to complain.
- Rewriting assumes we know better and are willing to do better - fundamentally most organizations don’t,
and that’s how they got in the mess in the first place.
- Must identify and fix the behaviors that let things rot.
- Honesty is critical - if you have cultural obstacles to improvement, you’ll probably repeat your same mistakes.
Don’t blindly derive requirements from the old
Don’t accept stupid http://www.theregister.co.uk/2016/09/07/linus_torvalds_wont_apply_shtforbrains_stupid_patch/
- Stability first
- Find the fragile parts and fix them, otherwise they will eat you alive as you try to change more
- get what you have under control/predictable in terms of patching, etc.
- deployment and production problems are clues to help you find the fragile parts.
- Deployments have to become non-event
- Watch your step
- Try to keep units of work small, very small, especially while you’re learning
- If it turns out you took the wrong fork in the road, you want to know before you drive half way across the country.
- Avoid repeating complexity you don’t understand (“what’s the business driver?!”)
- It’s OK to rewrite something crappy if you understand it and document
- It’s not OK to bring things into your new code base that you don’t understand
If you do understand, for god’s sake document it in the code. These are the cases where one should comment their code!
So we have to rewrite
- Find a way to do it gradually
- this is where web applications are great. It’s easy to coordinate too web apps fairly gracefully
- Add a front-end server, route traffic to the two apps by URL pattern, get basics of session shared
- You can even do this by having one app make a web service call to the other.
- It’s really like agile for your least agile product
- Complete version control
- 12 factor app, first point - distributed system vs one application
- eliminate differences between environments
- Sidebar: process maturity
- Cannot outrun process immaturity
- Lack of change control at any point in the stack will kill you
- Heroes hide problems
- quick fixes prevent prevention
- “Quick fixes” prevent root cause analysis
- prevent developing the knowledge you need to make better engineering decisions going forward
- The people doing this often think they are helping provide good customer service, but they’re really doing the opposite.
- The only good call to the help desk is the one that never happens
- Ad hoc fixes must be accompanied by follow up to find real issues
- Build a wall
- You have to have some core where you don’t compromise on standards.
- If you don’t, things rot very quickly
- But what if we can’t fix some things today?
- Make an abstraction to hide it
- Keep the mess out of the core
- Only code to the new standards
- Remove the shim later
- This also gives you a way to learn about the right fix before you commit to revising a lot of the legacy code
- The right way to undo the legacy problems is often not nearly as clear as how you want to move forward. Conversion is hard.
- Find out what’s important and what gets used
- Unraveling and porting something, only to find nobody cares really sucks
- This can be hard to retrofit into the legacy system in a useful way, but it’s absolutely worth it.
- Make sure you can tell real users vs your internal testers
- The easiest legacy feature to deal with is the one you kill
- Remove dead stuff - trust your version control
- Who leaves dead code around comment out?
- Story about how I tolerated this until I spent half a day reformatting some really awful code only to figure out it was 95% commented out
- Find dead DB objects and remove them
- Not doing this is like looking for clean dishes in a hoarder house. You can do it, but it makes it a lot harder.
- Develop tools to help find these things, grep is your friend. Run it regularly, as you prune things, it’ll be amazing what becomes orphaned
- Don’t tolerate mysteries
- You got into this mess because of things that were allowed to rot
- Anything that requires troubleshooting once will need it again
- Make it easier for next time so you don’t have to keep paying the same price
- We can’t always get it fixed in one shot, but always improve something
- Often this is a really bad cultural problem, but you’ve got to break the back of it if you want to get rid of the support problem.