Modern Observability with OpenTelemetry: The Standard That's Changing Everything

Before OpenTelemetry, observability instrumentation was a vendor lock-in problem. You’d instrument your code with Datadog’s SDK, then Datadog owned your telemetry data. OpenTelemetry changes this: one vendor-neutral SDK, exportable to any compatible backend.

What OpenTelemetry Is

OpenTelemetry (OTel) is an open-source observability framework providing: a specification for telemetry data (traces, metrics, logs), SDKs in every major language, and a collector component for processing and forwarding telemetry to backends.

The critical benefit: once you instrument with OpenTelemetry, you can send your telemetry to Datadog, Grafana, Honeycomb, Jaeger, or any other compatible backend — without changing your application code.

Implementing Traces

Distributed tracing is where OpenTelemetry delivers the most immediate value. In a microservices environment, a single user request might touch 10 services. Without tracing, when something is slow, you have 10 sets of logs with no clear connection. With tracing, you see a waterfall of spans showing exactly where time was spent.

Auto-instrumentation (available for Node.js, Python, Java, .NET) instruments common frameworks automatically without code changes.

Choosing Your Backend

• Grafana stack (Tempo + Prometheus + Loki): Open source, self-hostable. High operational overhead. Best for teams with infrastructure capacity.
• Honeycomb: Purpose-built for high-cardinality observability. Excellent for complex distributed systems.
• Datadog: Comprehensive APM with native OTel ingest. Expensive but polished and full-featured.
• Grafana Cloud: Managed Grafana stack. Good price/performance for teams that want Grafana tooling without self-hosting.

Instrumentation Coverage Is a Gradual, Prioritized Process

Organizations adopting OpenTelemetry frequently face the question of how comprehensively to instrument an existing codebase, and attempting full instrumentation coverage immediately is rarely the right approach. More effective adoption strategies prioritize instrumentation coverage based on actual operational pain points — starting with the services most frequently involved in production incidents or the request paths most critical to core business functionality — rather than pursuing uniform coverage across an entire service fleet simultaneously. This prioritized approach delivers observability value where it matters most early in the adoption process, building organizational confidence and demonstrated value that supports the case for continued investment in broader instrumentation coverage over time.

Cardinality Management Prevents Cost Surprises

A specific operational challenge that catches many teams off guard when scaling OpenTelemetry adoption is metric and trace cardinality — the number of unique combinations of label or attribute values associated with telemetry data. Adding a high-cardinality attribute like a user ID or request ID directly as a metric label, rather than keeping it only in trace data where high cardinality is expected and manageable, can cause metric storage costs and query performance to degrade dramatically as the number of unique label combinations multiplies. Teams new to OpenTelemetry frequently encounter unexpected cost spikes from cardinality issues before developing the intuition for which attributes belong in metrics versus traces, making this one of the more common and costly early mistakes in observability instrumentation.

Correlating Logs, Metrics, and Traces Effectively

The genuine power of a unified observability approach comes from the ability to correlate across the three pillars — jumping from an anomalous metric spike directly to the relevant traces and logs from the same time window and request context — rather than treating logs, metrics, and traces as separate, disconnected data sources requiring manual correlation by an engineer during an incident. Achieving this correlation requires consistent context propagation throughout the instrumentation — ensuring trace IDs are included in log output and that metrics carry sufficient labels to connect back to relevant traces — a level of instrumentation discipline that delivers substantially more value during incident investigation than collecting the three telemetry types independently without this deliberate cross-referencing capability built in from the start.

This article is part of our ongoing coverage of Cloud & DevOps. For related reading, see DORA metrics and engineering velocity and platform engineering.