The UI for Cloudwatch Logs completely crumbles if you use it for Lambda functions, especially if you have a lot of executions.

We run ETL pipelines and have lambda as a component of that, maybe 250k-500k invocations a day.

If you want to search through the logs for a particular invocation at a particular time (e.g. when an error happened) the UI just cannot handle it.

I think part of the reason is Lambda creates a new "log stream" every time a new container is created, which I'm guessing is how they shard the logs. So you end up with hundreds/thousands of log streams within your lambda log group and the backend has to search them to find the particular search pattern you are requesting.

Most of the time we have to export the logs to S3, download them and do some manual grepping.