Gradually migrating to typescript across multiple js codebases with a mid-size team

Photo by Chris Lawton on Unsplash

Disclaimer: This is more like a transcript of a talk that I gave internally at my company, so the language might be abridged at some places.

Glossary:

If you are not a frontend developer, it might be helpful for you to have an understanding of these terms as these would be mentioned multiple times in the article.

Storybook:

A tool for visual documentation of UI components. We use this in our codebases to document the component and its props and to allow experimenting with the different UI states of the component.

Mono repo:

A way of organizing multiple packages in a single repository. We use this setup with lerna for our shared component library.

Shared codebase

We have a shared component library as part of our basic design system. This is primarily maintained by the smallcase FE team but other FE teams also contribute to and are consumers of it. This is a mono repo setup which generally has a different independent package for each shared component.

Prop Types:

A way to add meta data to React component props, so that some basic validations (related to type of the prop, or even some logic) are automatically carried out at runtime in development mode. The violations are then logged to the browser console.

Type definitions:

A way to expose type information for code not written in typescript, so that typescript enabled codebases can still make use of the type information.

What to expect:

• contents of this article would be more about the process and workflow around integrating and migrating to typescript, and less about the technical aspects of typescript
• overall understanding of how we undertake bigger engineering tasks in FE team
• general understanding of FE codebase architecture, and where typescript fits in
• a team perspective of how to include typescript in the workflow

What not to expect:

• any major pointers about whether you should choose typescript for your workflow or not
• discussions about technical aspects of typescript
• a developer perspective of what typescript enables in the codebase

A quick note about our engineering workflow

• Gradual adoption
• Enough space and time for experimentation before we make something mandatory
• Prep for the next set of engineering goals starts from the previous quarter.
• Wait for conventions and patterns to manifest, document them and then enforce them
• Once mandatory, it is enforced for all new development
• make the existing code compliant when picking up changes on it in future iterations
• Experimentation starts from less critical codebases or pieces of code, and is nonblocking

Why did we plan to move to typescript?

DX:

• we were already using prop types in the codebase for documenting a component and its contract, and vscode intellisense does not work the best with proptypes out of the box, so we needed something better
• we had been using JSdoc in a few places, and the intellisense was great.
• vscode automatic inference for types was very helpful

Experiment:

• shiny new thing which everyone wanted to try out, specially when the developer ecosystem around js / react wouldn’t stop talking about it
• some devs had background in working with typed languages and were positive about the impact

Documentation:

• With a lot of new devs in the team, it would be really helpful to have some in-code documentation about what a component accepts, what the backend api response looks like, especially when working on existing code. This would help devs to understand the data types much better. Having types for the data and the components serves as another bit of helpful documentation.

Other reasons:

• prop types works at run time and does not have a compile time IDE integration. We did not have a good workflow for checking the prop types warnings in the browser console after running the code, so it wasn’t that helpful.
• early feedback / compile time errors

Things to consider before migrating:

• The whole team cannot upskill at once
• This is where typescript shines, as you can choose to enable it piece by piece
• Our existing codebases are too complex to convert to ts in one go.
• We were confident about the capabilities of typescript, but we were not completely convinced whether the effort to learn typescript for the whole team would be worth the benefits it brings, so we wanted to start small.
• we wanted to enable ts in an existing js codebase, so we did not enable the strict mode by default
• We want to add types, but we wouldn’t want to essentially change how we write code. For example, we wouldn’t suddenly change our patterns to align with object oriented paradigm.
• we wanted it to be non-intrusive and non mandatory, so it would be ok to have incorrect types or any type for things that you can't type write now
• shouldn’t be an added chore to add / maintain types if they are not being used for anything helpful
• no complex types to be introduced unnecessarily
• one off code does not need to be strictly typed
• coverage is not something we would consider a success parameter

The migration journey:

Experimenting with typescript in one-off independent codebases

This setup is very convenient for running isolated experiments without jeopardizing or blocking the primary codebases. This is a good compromise because we still get to try things in prod like setups, but with less impact. This is how we generally approach a log of bigger changes as part of our engineering plan. There were already other engineering tasks on priority in main codebases, and as per our workflow, we start introducing things which are planned for the next set of engineering goals, but not make them mandatory yet, so we started with ts experimentation in some of our independent codebases.

Learnings:

• basic types and basics of type definitions
• figured out workflow issues related to co-locating types with code, but also reusing them at multiple places
• we ended up with keeping the types in a separate file co-located with the component, but the types themselves were global and were not needed to be imported
• this was done even for non-reusable types, and made the setup slighltly prone to global namespace naming conflicts, though we did not get into any such issues as the codebase was small and the types were limited.
• ambient modules vs module, import / export
• how babel transpilation can work irrespective of it not understanding types information

Enabling typescript in one of our regular production codebases

This was done in parallel to the previous experiment. We have a nextjs codebase for a server rendered application. Nextjs provides typescript support out of the box, we can enable it on need basis, so there was minimum effort to enable typescript in that codebase, and there was no need to actually understand the complexities of ts config yet.

• we wanted to focus on what we can do with ts in code rather than learning and tinkering with the config from scratch
• enabled for js files, and not strict

Learnings:

• How typescript-eslint rules conflict with existing eslint config in our setup
• Basics of how redux actions / reducers can be typed with javascript
• Basics of how to use js and ts together and how well they work

Handwritten type definition files in shared codebase

After the previous experimentation, we were convinced that typescript is worth experimenting with in our primary dev workflows in our main codebases. So we started exposing the types for our shared component library by using hand written type definition files for each individual package.

We did not want to start writing the new code in typescript and instead use type definition files because:

• although we were the primary maintainers, the shared codebase was consumed and updated by other teams as well, who wouldn’t have wanted to jump on to the ts bandwagon just yet. So, to make sure that the codebase is still readable for them and ts consumers can consume types we went ahead with this approach. This was also non intrusive and auxiliary to the actual code, so it would give
• our shared component library is built on the consumer instead of having a build step of its own, because of how certain packages are written, so adding typescript here would have required all the consumers to include typescript in their build setup so that the shared component library could be built.

We went for handwritten type definition files because:

• We did not know how well the auto generation works (we will discuss this in a while)
• We wanted devs to learn typing from scratch, and then move to auto generation

This was a fairly non-typical setup and there are not a lot of examples online, especially with how we wanted to handwrite and maintain types co-located with the code.

Learnings:

• For libraries, the general convention is to have one single types file per package, as there is only one entry point for types per package, but some of our packages had subfolders. We wanted to co-locate type definitions with each individual module (not the whole package), so we figured out how to co-locate types with internal folders, and then re-export them from the package level type definition file
• Since this was a shared codebase, there are certain dependencies on always having a consumer to test the changes that happen in the shared codebase. To workaround this, we added a storybook consumer in the same shared library codebase, to verify that types are working fine at the time of adding them in the codebase.
• type definition files are not type checked by default in the tsconfig, needed to enable
• started writing stories (storybook consumer) in typescript. This was enforced later.
• difference between DefinitelyTyped vs typings as part of the code
• Figured out how typescript discovers type definition files
• Started with the type safety conventions doc (improved it further with the next iterations)

JSDoc to generate types in non typescript codebases:

• One of the easiest entry points to typescript if you are working with a ts enabled IDE
• vscode is written in typescript and works very well with ts. One of the things that it provides out of the box is automatically inferring ts types from a lot of compile time information; jsdoc comments and types being one of them.
• Though we were already writing JSDoc comments for prop types, for them to appear correctly in storybook Docs panel, later we enforced JSDoc comments in our shared utils functions within the codebase.

Learnings:

• JSDoc does not support all usages of typescript: advanced types, generics etc
• Once we move to typescript, we would want to move all the JSDoc types to typescript types

Using typescript in JSDoc

• Once we enforced JSDoc for shared functions, and utils functions within individual codebases, we figured out that the intellisense is great but JSDoc in itself is not able to cover all the typescript usages
• vscode allows using typescript directly in jsdoc. so we figured that in non ts enabled codebases, since we have already integrated jsdoc, we will just start defining type definitions locally and reference them in the jsdoc comments. This was possible due to vscode working well with ts by default, we didn’t need to even have a tsconfig.
• This will help us transition to typescript more easily in these codebases in future, as we would already have the typescript types defined, and we would just need to rearrange them if at all required

Learnings:

• reduces code readability slightly as types are now in a separate d.ts file, away from the usage

Enforce typescript for any newly added code in ts enabled codebases

• Wherever possible, we made it mandatory to write any new code in typescript. This was fine even for the people who had recently joined the team, as there were enough usage examples in the codebase already.
• It wasn’t possible to do this in the shared codebase just yet, because the consumers were consuming the source code, and the source code needs to be in js, if the consumers haven’t enabled ts in their build config. So, for the shared codebase, we enforced it for any code being added which wasn’t to be consumed by the mono repo external consumers. We enforced it for tests and stories within the shared codebase.

Shared types package

• We realized that there are a lot of common types that we use across the web and app codebases and the shared component library, so, in the shared codebase, we created a types package for shared types. Consumers can consume it as any other package (even use it for defining the jsdoc types in consumer codebases)

Start using typescript in some packages in the shared component library and generate javascript as output:

• As discussed earlier, it was not possible for us to have a build step for the whole of the monorepo to create a distribution for every package, which could be consumed on the consumer as a javascript package, as there are a few packages which depend on build time configuration on the monorepo consumer side. We are planning to re-architect this in future so that there is no build time dependency, but for now, we realized that we could still enable the build step for individual packages, so that we can generate the javascript distribution for consumers and use typescript in the actual package.
• This is also easier to do now, as now we have enforced ts in the storybook consumer, so any contributors to the shared codebase are already aware of typescript, so shouldn’t be a problem for them to understand the ts source code inside the packages.

Automatically generate base types and use them to create proper types

• typescript compiler allows generating type definitions from a lot of existing conventions like JSDoc and through automatic inference.
• There is a command to generate type definitions for a source file using tsc.
• For proper type definitions to be generated, there have to be proper types specified in the source code in some way for example, in jsdoc, bit even without that the inferred type information can be used as a base to create proper types instead of creating them from scratch by hand.

Enable typescript on other codebases with better config

• in the non ts codebases, we have started enabling typescript taking all our learnings from the experiments

strict mode

• Enforce strict mode so that any type is not allowed

lint on running tests

• we will run tsc on the staged files on every commit to ensure that type errors are fixed before code is committed.

What has worked:

• enforces explicit documentation, which leads to an implicit understanding of external contracts (for example api responses)
• share-ability across cross platform codebases
• error triaging is faster
• A lot of runtime checks and unit tests are eliminated
• feedback loop during development is quicker
• the workflow is not pedantic, so easy to adopt
• This was taken up as an assignment to improve the code understanding and make it more compile-time safe, rather than to learn typescript, so we were able to manage expectations and complexity well. For example,
• we did not introduce or push for advanced types or generics if they hampered the readability or code understanding.
• we did not unnecessarily start writing object oriented code just because we have typescript

What has not worked:

• typescript as an independent language is too powerful but too vast and complex, as we go ahead we would get into situations where some devs would start using derived types, and other advanced types, and many of the other devs would struggle with understanding this complex code.
• inline types make the code less readable, so we would need to be more cautious with the conventions around them. We would ideally want someone who only knows js to be able to read the code and understand it, without a lot of noise from the inline type information.

Further plan and improvements:

• the current setup is very helpful for devs who use a piece of code, but not a lot for devs who just read it, without an IDE for intellisense (in github or shared component library for example). This needs to improve and we need better patterns for co-locating types with the code without impacting the readability
• enable typescript in the app codebase
• share types with api codebase or auto generate types for FE from api definitions (For example, joi schema to ts)
• We currently do not enforce typescript and proper types, devs ignore warnings on the console. For many devs, it is more of a hint to the developers than a contract. We would fix this by making sure that builds fail if there are type errors.
• duplicate effort in creating d.ts files specially when you have to define the const map as a enum again in ts. Automatic type generation should help with this
• redundant documentation at different places — proptypes, JSDoc, typescript types
• improve existing patterns using inferred types, advanced types, derived types which auto update
• move to typescript as default in all codebases
• maintain and add to the conventions, and publish them externally

More resources:

• https://betterstack.dev/blog/publishing-type-definitions-with-npm-package/#heading-generate-type-definitions
• https://gils-blog.tayar.org/posts/jsdoc-typings-all-the-benefits-none-of-the-drawbacks/