Not OP, but we have a very similar setup. We use https://github.com/vega/ts-json-schema-generator in a prebuild hook to generate a schema from Typescript. The schema is then fed into Fastify, and referenced in route configurations for incoming (validation) and outgoing (fast serialization) data, as well as auto-generated Swagger. Took some custom code to simplify attaching definition references to our route configs, but it works quite well.

As an extra, our frontend uses the same shared TS code for data transfer objects, this way there's an extra level of type safety at the API.