NuttawutBlogพูดถึง

Trace application workflows by Jaeger

This time, I will use Jaeger for tracking processes of my Go server. I will use a Jaeger Go library from OpenTelemetry to connect to jaeger collector

Then create a Go module first

mkdir jaeger-tracing
cd jaeger-tracing
go mod init jaeger

Then write a tracer provider

touch provider.go

package main

import (
	"go.opentelemetry.io/otel/attribute"
	"go.opentelemetry.io/otel/exporters/jaeger"
	"go.opentelemetry.io/otel/sdk/resource"
	tracesdk "go.opentelemetry.io/otel/sdk/trace"
	semconv "go.opentelemetry.io/otel/semconv/v1.7.0"
)

var (
	_serviceName = "myservice"
	_environment = "production"
)

func NewTraceProvider(url string) (*tracesdk.TracerProvider, error) {
	exporter, err := jaeger.New(jaeger.WithCollectorEndpoint(jaeger.WithEndpoint(url)))
	if err != nil {
		return nil, err
	}
	provider := tracesdk.NewTracerProvider(
		tracesdk.WithBatcher(exporter),
		tracesdk.WithResource(resource.NewWithAttributes(
			semconv.SchemaURL,
			semconv.ServiceNameKey.String(_serviceName),
			attribute.String("environment", _environment),
			attribute.Int64("ID", 1),
		)),
	)
	return provider, nil
}

This NewTracerProvider function returns a provider that connects with the Jaeger Collector endpoint with some simple config.

Now, write the main function to call this function

touch main.go

func main() {
  provider, err := NewTraceProvider("http://localhost:14268/api/traces")
  if err != nil {
    log.Fatal(err)
  }
  defer func() {
    ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
    defer cancel()
    if err := provider.Shutdown(ctx); err != nil {
      log.Fatal(err)
    }
  }()
  otel.SetTracerProvider(provider)
}

Then start wrting an API server, continue writing the below code in the main function

func main() {
  // ...
  ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
  defer stop()

  r := gin.Default()
  SetupHandler(r)
  srv := &http.Server{
    Addr:    ":8080",
    Handler: r,
  }

  go func() {
    if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
      log.Fatal(err)
    }
  }()

  log.Println("Server is running, press ctrl+C to stop")
  <-ctx.Done()
  stop()
  ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
  defer cancel()
  if err := provider.Shutdown(ctx); err != nil {
    log.Fatal(err)
  }
  if err := srv.Shutdown(ctx); err != nil {
    log.Fatal("Server is forced to shutdown: ", err)
  }
  log.Println("Server exiting")
}

Next, implement SetupHandler function to register an endpoint for the API server

touch controller.go

package main

import (
	"context"
	"time"

	"github.com/gin-gonic/gin"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/attribute"
)

func FindItem(ctx context.Context) {
	tr := otel.Tracer("component-query")
	_, span := tr.Start(ctx, "query")
	defer span.End()
	span.SetAttributes(attribute.Key("search.id").String("a1b2c3d4"))
	// do something
	time.Sleep(time.Millisecond * 500)
}

func Filter(ctx context.Context) {
	tr := otel.Tracer("component-filter")
	_, span := tr.Start(ctx, "filter")
	defer span.End()
	// do something
	time.Sleep(time.Millisecond * 100)
	// code example when there is an error
	// span.SetStatus(codes.Error, "fail to filter items")
	// span.RecordError(errors.New("item format is not correct"))
	time.Sleep(time.Millisecond * 200)
}

func SetupHandler(r *gin.Engine) {
	r.GET("/", func(c *gin.Context) {
		tr := otel.Tracer("component-http-request")
		ctx, span := tr.Start(context.Background(), "http-request")
		FindItem(ctx)
		Filter(ctx)
		defer span.End()
	})
}

From the above code, I have one endpoint which is the root path (GET "/"). When there is a request coming, I create a tracer named component-http-request and then start the span while running two functions (FindItem and Filter). In those two functions, I wrote sleep functions to mock API behavior. The functions also create tracers that would be childrens of the main tracer (component-http-request).

Before starting the server, let's prepare Jaeger with docker and then run the Go server

docker run -d -p 14268:14268 -p 16686:16686 jaegertracing/all-in-one
go run .

You can visit jaeger webserver in localhost:16686. In the webserver, at the form in the left side, set Service to myservice and click Find Traces. There is no traces to inspect yet; let's add some by doing http request to the Go server

curl localhost:8080

Then refresh the jaeger webserver and you will see a new trace has been added. Click the trace to inspect and see what happened.

There is a bar graph that tells about when an event was started and when it was ended. If there is error, it can be used for debugging since it tells when and where the error happened; this would reduce your time to find the bug.

sources