(workflows) Brainstorm ideas for waterfall span comparison

[Mitan]

Brainstorm ideas on how the Cohort comparison approach can be extended to waterfall comparison. From PRD:

Milestone 2 - Span Waterfall Comparison

User Story

• As a developer, I want to quickly identify if the p95 duration of my checkout page is longer due to code paths taking longer or due to additional processing taking place compared to the p50 so that I can quickly identify areas of improvement
• As a developer, I want to identify what code paths are causing my server API calls to take longer for EU compared to the US region

Acceptance Criteria

1. The user should be able to specify an outlier and a baseline cohort explicitly or implicitly via the design. This could be done in several ways
   1. Outliers in heat maps/scatter plots
   2. Highlighting a regressed area of a metric chart
   3. From the table results of aggregations
   4. From a particular trace view?
2. Compare counts and durations of spans in one cohort vs another
   1. Is a specific span being executed more in one group the other
   2. Is a specific span taking longer in one group vs the other (here, it would be so awesome to have exclusive time)
3. Plot counts/percentiles of these spans