Skip to main content

Self-Hosting Guide - Log Analyser

Welcome to the Jitsi Meet Log Analyser project! This integration leverages Grafana Loki and OpenTelemetry to enable effective log management and analysis for Jitsi Meet components.

Overview​

This project offers a streamlined setup for collecting, parsing, and visualizing log data generated by Jitsi Meet components. The integration includes:

  • A Docker Compose setup file (log-analyser.yml) for configuring Loki and OpenTelemetry Collector.
  • A Docker Compose setup file (grafana.yml) for configuring Grafana.
  • A unified Docker Compose command that initiates all services concurrently.
  • Instructions on configuring Grafana with Loki as a log data source for enhanced visualization and analysis.

Getting Started​

Prerequisites​

Before you begin, ensure you have the following installed on your system:

Setup​

To set up the project, follow these steps:

  1. Clone the repository:

    Clone the Jitsi Meet repository that contains the necessary Docker Compose files.

    git clone https://github.com/jitsi/docker-jitsi-meet.git
    cd docker-jitsi-meet

  2. Update Jitsi Meet Docker Compose Configuration:

    To enable log collection and analysis, you need to modify the docker-compose.yml file for Jitsi Meet components. Add the following configuration to each Jitsi service within the docker-compose.yml file:

    logging:
      driver: "json-file"
      options:
        labels: "service"

    This configuration ensures that logs are collected in JSON format and tagged with service labels, which is essential for Loki to properly ingest and index the logs.

  3. Start the Docker containers:

    Use the following command to start all required services, including the Jitsi Meet components, Grafana, Loki, and OpenTelemetry:

    docker-compose -f docker-compose.yml -f log-analyser.yml -f grafana.yml up -d

    • Explanation:

      • The command combines multiple Docker Compose files to launch the entire stack:
        • docker-compose.yml launches the Jitsi Meet components.
        • log-analyser.yml sets up the log analysis tools, including Loki and OpenTelemetry.
        • grafana.yml initializes Grafana for log visualization.
      • Logs generated by the Jitsi Meet components are automatically forwarded to Loki via the OpenTelemetry Collector, making them available for analysis in Grafana.

    • Note: If you only want to use Grafana for log visualization without the log analysis tools, you can run grafana.yml independently. However, for the complete log analysis setup, both log-analyser.yml and grafana.yml are required.

Access Grafana​

After starting the services, follow these steps to access Grafana:

  1. Open Grafana in your web browser:

    Navigate to http://localhost:3000 to access the Grafana interface.

  2. Log in to Grafana:

    Use the default login credentials provided below:

    • Username: admin
    • Password: admin

    (It is recommended to change these credentials after the first login for security purposes.)

Pre-configured Dashboards​

Grafana comes with several pre-configured dashboards specifically designed for monitoring different Jitsi Meet components. These dashboards will be automatically available once you log in to Grafana.

Important❗️: For data to appear in these dashboards, logs need to be generated by the Jitsi Meet components. Here are the available dashboards:

  • Jicofo Dashboard: Visualizes logs related to Jitsi Conference Focus (Jicofo), which handles media and signaling in Jitsi Meet.
  • JVB Dashboard: Focuses on Jitsi Videobridge (JVB) logs, showing details on video streaming and performance metrics.
  • Prosody Dashboard: Monitors Prosody, the XMPP server used by Jitsi Meet for signaling.
  • Web Dashboard: Displays logs and metrics related to the web frontend of Jitsi Meet.
  • Jitsi All Dashboard: A comprehensive dashboard that aggregates logs from all Jitsi Meet components.

Jitsi Meet Log Analyser Dashboard

Filtering Logs with LogQL​

Beyond the pre-configured dashboards, you can explore and filter logs in Grafana's "Explore" section using LogQL, a powerful query language designed for Loki. Here’s how you can filter and analyze logs:

  1. Access the Explore section:

    In Grafana, navigate to the Explore tab from the left sidebar. This section allows you to run queries on your logs in real-time.

  2. Select the Loki Data Source:

    In the Explore section, ensure that the Data Source is set to Loki. This is essential because Loki is the backend that stores and manages the log data collected from Jitsi Meet components.

  3. Using LogQL for Filtering:

    LogQL enables you to create complex queries to filter specific logs. For example, the following LogQL query filters logs for the jitsi-jicofo component and parses them:

    {exporter="OTLP"} | json | attributes_attrs_service="jitsi-jicofo"
    • Explanation:
      • {exporter="OTLP"}: Selects logs that are exported via OpenTelemetry.
      • | json: Parses the log data as JSON, making attributes accessible for filtering.
      • attributes_attrs_service="jitsi-jicofo": Filters logs where the attributes_attrs_service field equals "jitsi-jicofo".

    You can modify the query to filter logs from other components or adjust the criteria according to your needs.

    For more details and advanced usage of LogQL, you can refer to the official LogQL documentation from Grafana.

    LogQL Query Example

Usage​

Once the setup is complete, you can start exploring your log data in Grafana. The pre-configured dashboards provide an insightful visualization of the logs collected from Jitsi Meet components. Use these dashboards to:

  • Parse Logs: View detailed logs collected from various components.
  • Visualize Logs: Analyze log data through various charts, graphs, and panels to gain insights into system performance and issues.

Customizing Dashboards​

While the pre-configured dashboards provide a solid starting point, you may want to customize them or create new dashboards to suit your specific needs. Here's how you can do that:

  1. Create a New Dashboard:

    • Go to the Grafana home page, click on the "+" icon on the left sidebar, and select "Dashboard."
    • Add panels to visualize different metrics or logs by selecting the appropriate data source (Loki) and using LogQL queries.

  2. Customize Existing Dashboards:

    • Navigate to an existing dashboard and click on the "Edit" button (pencil icon) on any panel.
    • Adjust the LogQL query, visualization type, and panel settings to match your requirements.

  3. Save and Share Dashboards:

    • After customizing, save the dashboard. You can also export it as a JSON file to share with others or for backup.
    • To export the dashboard:
      • Click on the dashboard title to open the options menu.
      • Select "Dashboard settings" > "JSON Model".
      • Click "Download JSON" to save the file locally.

  4. Contribute to Jitsi Meet Dashboards:

    • If you've created or customized a dashboard that could benefit the wider Jitsi community, you can contribute by updating the relevant dashboard JSON file.
    • Here’s how to do it:
      • Export the JSON file of your customized dashboard as described above.
      • Locate the corresponding Jitsi component dashboard JSON file in the repository (e.g., jicofo-dashboard.json, jvb-dashboard.json).
      • Update the existing JSON file with your changes.
      • Submit a pull request to the repository with the updated JSON file and a brief description of the changes you made.
    • This helps improve the pre-configured dashboards and makes your contributions available to all users.

  5. Grafana Provisioning:

    • The Jitsi Meet Log Analyser uses Grafana provisioning to manage dashboards. When you update a dashboard JSON file in the repository, it will be automatically provisioned in Grafana when the stack is deployed.
    • This ensures that everyone using the repository gets the latest version of the dashboards without needing to manually import them.

By following these steps, you can not only customize your own monitoring setup but also contribute improvements back to the Jitsi community.

Troubleshooting​

If you encounter issues while setting up or using the Jitsi Meet Log Analyser, here are some common problems and their solutions:

  1. Grafana Not Starting:

    • Check if the Grafana container is running with docker ps and inspect logs using docker logs grafana for any errors.

  2. No Logs in Grafana Dashboards:

    • Ensure that Jitsi Meet components are generating logs. Clear browser cache, reload Grafana. Ensure OpenTelemetry, Loki, and Grafana containers are all running with docker ps, and inspect each container's logs for issues using docker logs <container_name>.

  3. OpenTelemetry Collector Not Forwarding Logs:

    • Check OpenTelemetry's logs with docker logs otel, ensure it's connected to the correct endpoints, and verify the log format is correct.

  4. Authentication Failures in Grafana:

    • Restart Grafana with docker restart grafana otel. If still unsuccessful, delete the data volume with docker-compose down -v and restart to reset to default credentials (admin/admin).

  5. Slow Queries:

    • If LogQL queries are slow, try optimizing the query in Grafana.

  6. Permission Issues:

    • If you encounter permission issues, make sure that Docker has the necessary access rights to the directories where logs are stored.

  7. Docker Network Issues:

    • Verify Docker network connections, IP range, and restart the network if necessary.

  8. OpenTelemetry Collector Not Forwarding Logs:

    • Check OpenTelemetry logs, verify configuration, and ensure log format compatibility.

  9. Docker Containers Failing to Start:

    • Use docker-compose logs to view detailed startup errors, and check for common issues like incorrect configurations.

Acknowledgements​

We appreciate your interest in the Jitsi Meet Log Analyser project! If you encounter any issues or have questions, feel free to reach out to the project maintainers or contribute to the repository.