Loading…
Edinburgh, Scotland, UK
October 21 & October 25 | Co-Located Events, Tutorials, & Workshops
October 22-24 | Conference
Find out more information for Open Source Summit + Embedded Linux Conference & OpenIoT Summit Europe 2018

Please note that you can view and download presentations on the Open Source Summit and Embedded Linux Conference + OpenIoT Summit slides pages. 
Back To Schedule
Wednesday, October 24 • 16:15 - 16:55
Scalable Monitoring of Apache Spark with Prometheus - Diane Feddema & Zak Hassan, Red Hat

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
Feedback form is now closed.
As Apache Spark applications move to a containerized environment, there are many questions about how to best observe and configure clusters in the container world. In this talk we will demonstrate a set of tools to better monitor performance, alert on symptoms and identify optimal configuration settings. We will demonstrate how Prometheus, a project that is now part of the Cloud Native Computing Foundation (CNCF: https://www.cncf.io/projects/), can be applied to monitor a cluster and send alerts in a containerized spark environment.

In our examples, we will gather spark metrics output through Prometheus and present the data with Grafana dashboards. We will use our examples to demonstrate how performance can be enhanced through different tuned configuration settings. Our demo will show how to configure settings across the cluster as well as within each node.
Speakers
avatar for Diane Feddema

Diane Feddema

Principal Software Engineer, AI/ML Performance on RHEL and OpenShift Operator Development, Red Hat
Diane Feddema is a principal software engineer at Red Hat Inc, in the Performance and Scale team. Diane is currently focused on developing and applying machine learning techniques for performance analysis using hardware accelerators, automating these analyses and displaying data in... Read More →
avatar for Zak Hassan

Zak Hassan

Senior Software Engineer - AI/ML CoE, CTO Office, Red Hat Inc.
Currently focused on developing analytics platform on OpenShift and leveraging Open Source ML Frameworks: Apache Spark, Tensorflow and more. Designing high performance and scalable ML platform that exposes metrics through cloud-native technology: Prometheus and Kubernetes.

Wednesday October 24, 2018 16:15 - 16:55 BST
Fintry Auditorium, Level 3
  Artificial Intelligence & Data Analytics
  • Experience Level Any

Attendees (67)