Centralized Logging Solution for AMRIT's Distributed Infrastructure

  • 02 Mins read
  • DevOps

CHALLENGE: Distributed Logging Complexity

AMRIT’s infrastructure—spread across multiple servers located nationwide—presented significant logging and monitoring challenges. With servers dispersed across the country, tracking system performance, identifying issues, and maintaining comprehensive system visibility became increasingly complex. The need for a centralized, effective logging solution became paramount to ensure system reliability and quick troubleshooting.

SOLUTION: ELK Stack Logging Infrastructure

Over the course of implementing the logging strategy, we successfully developed a comprehensive centralized logging solution using the ELK stack. This solution included:

Key Objectives:

  • ECS Logging Configuration: Implemented standardized logging across all API services using Elastic Common Schema (ECS) for consistent log formatting.

  • Log Collection Infrastructure: Deployed Filebeat and APM agents to collect logs and traces from distributed servers, enabling comprehensive system monitoring.

  • Source Identification: The logs and traces were injected with an environment property to precisely identify log sources across multiple server locations.

  • Intelligent Log Management: Implemented an effective log sharding strategy to optimize index management and improve query efficiency, with a log retention policy for systematic data pruning and storage management.

Expected Outcomes:

The implementation of the centralized ELK stack logging solution for AMRIT is expected to have the following outcomes:

  • Enhanced Visibility: Provide a single, comprehensive view of logs across distributed services, enabling rapid issue identification and resolution.

  • Improved System Reliability: Enable faster troubleshooting and performance monitoring across AMRIT’s nationwide server network, with APM traces providing deep insights into query performance, latency metrics, error tracking, and application bottlenecks.

  • Secure Access Control: Role based access control policy controls log and trace visibility for development teams.

About the Contributor

Ivor D’Souza is a Software Engineering masters student at the University of Limerick. His passion for DevOps, Data Engineering and Open Source technologies led him to contribute to AMRIT. With experience at Juspay and expertise in developing monitoring and logging solutions, Ivor is dedicated to improving healthcare accessibility.

Conclusion

The deployment of the ELK stack for monitoring on the AMRIT platform marks a pivotal step in enhancing operational efficiency and reliability. By automating log retention, centralizing data collection, and simplifying access to critical insights, we have significantly reduced manual overhead and improved system scalability. This setup not only ensures seamless monitoring across distributed servers but also empowers our teams to proactively address issues, contributing to the platform’s stability and performance. It demonstrates how thoughtful integration of technology can drive meaningful impact, enabling better service delivery at scale.