Skip to main content
Common operational issues, quick checks, and log locations.

​
Common problems and resolutions

502 errors - Chat API unreachable or unhealthy behind NGINX
  • Check service health: docker service ps <chat-api-service>
  • Review NGINX upstreams and logs for routing errors.
Kafka lag - Consumer slowdown or insufficient partitions
  • Inspect lag: kafka-consumer-groups --describe --group <group> --bootstrap-server <broker>
  • Increase partitions if needed: kafka-topics --alter --partitions <count> --topic <topic> --bootstrap-server <broker>.
Redis eviction - Memory pressure or wrong eviction policy
  • Inspect memory settings: redis-cli config get maxmemory and redis-cli config get maxmemory-policy
  • Apply an eviction policy such as redis-cli config set maxmemory-policy allkeys-lru.
TiKV region errors - Disk latency, contention, or store imbalance
  • Check cluster status: tiup cluster display
  • Rebalance if required: tiup cluster restart <cluster-name> --force.

​
Debugging commands

Container and Swarm diagnostics: 
docker logs <container>
docker service ps <service>
docker inspect <container>
TiDB cluster status: 
tiup cluster display
Kafka consumer diagnostics: 
kafka-consumer-groups --describe

​
Log locations

  • NGINX: /var/log/nginx/
  • Application logs: via docker logs per container
  • TiDB: /logs/tidb/
  • Kafka: /logs/kafka/