Suvodeep Pyne is a seasoned technologist and engineering leader with over 14 years of experience building scalable systems and driving technical innovation. Currently serving as a Founding Engineer and Lead at StarTree, Suvodeep has played a pivotal role in the creation and evolution of StarTree ThirdEye, a real-time anomaly detection platform built on Apache Pinot.
Under his leadership, the platform has grown from concept to production, delivering advanced monitoring and analysis capabilities to enterprise customers while generating a significant revenue stream for the company.
Led the development of StarTree ThirdEye, an anomaly detection platform, from inception to a full team. Shaped product strategy through customer engagement, enhancing revenue. Overhauled architecture for enterprise scalability and implemented Kubernetes readiness for reliable deployments.
Built Realia, a B2B educational platform with over 100,000 users and 35k+ monthly active users. Grew the team from zero to 15, acquiring 2,000+ clients. Developed a document scanner for students' notes, leveraging Java, React, and Firebase.
Optimized ML workflows and data pipelines as part of the Data Infra team. Led engineering for Azkaban, LinkedIn's workflow scheduler, enhancing scalability. Developed Cubert, a Hadoop-based computation engine, and a JDBC proxy server for SQL query delegation. Technologies used include Hadoop, Java, and Python.
Contributed to the GPU Driver Development Team by extending BIOS security and extraction modules. Developed C/C++ APIs for third-party vendors to debug BIOS errors, enhancing troubleshooting efficiency by 90%. Gained experience in kernel-level debugging on Windows and received a full-time offer.
ODesigning and implementing a web application for helping researchers perform various computations on their data using built-in functions that run on a distributed MATLAB framework.
Developed planning and routing algorithms for the Network Planning System, optimizing network layout and deployments. Recognized with two SPOT bonuses for outstanding performance during my tenure. Skills include Java and algorithms.
Responsible for maintaining and enhancing the FFS component, which manages the critical auto ticketing process. This component ensures e-ticket confirmation, oversees billing settlements with airlines, and updates PNRs in the Global Distribution System (GDS).
Developed an innovative algorithm to address the Traveling Salesman Problem using heuristics, achieving sub-optimal solutions with minimal error percentages. Created a MATLAB function library for processing dyadic tiles and calculating statistical properties like entropy and fractional dimension.
* indicates pending approval
The project involved independent validation of flight mechanics parameters for the SARAS aircraft using a MATLAB mathematical model. Conducted stability analysis under various flight conditions, successfully rectifying the model to support the development of an autopilot system.
Skills: Python
Activities and societies: programming team
Incomplete Matrix Profile-Based Anomaly Detection In Time Series Data, StarTree, CA
* indicates pending approval
Alert Monitoring Of Data Based On Recommended Attribute Values, StarTree, CA
Joins and aggregations on massive graphs using large-scale graph processing, LinkedIn, CA
StarTree ThirdEye is a real-time anomaly detection platform, built on top of Apache Pinot to offer unparalleled flexibility and power in monitoring and analysis. Designed to cater to the diverse needs of enterprise users, it is heavily customizable in identifying specific patterns or anomalies within data. It also allows you to construct derived metrics on the fly and feed into the anomaly detection pipeline seamlessly.
StarTree ThirdEye has a built-in suite of advanced anomaly detection algorithms. However, the platform is extensible so that users can add their own models, notification channels or authorization. This adaptability, combined with anomaly detection, notification and alerting, and root cause analysis tools, positions it as an essential tool for any organization aiming to automate their monitoring and alerting processes.
Learn more: https://startree.ai/products/startree-thirdeye
Cubert is a fast and efficient batch computation engine for complex analysis and reporting of massive datasets on Hadoop. The innovation here is the data preparation step that facilitates running better algorithms which perform an order of magnitude faster than off the shelf implementations that are available in the market.
Some of the algorithms used in the project have already been published in VLDB.
http://www.vldb.org/pvldb/vol7/p1462-vemuri.pdf
I am an active contributor to this project and have worked on several parts of the framework. Currently, I am working on the upcoming features to make Cubert support Incremental Computation.
Also, we have open sourced this project on GitHub. Please feel free to give it a try and let us know how it works for you.
Documentation for the project is available at http://linkedin.github.io/Cubert/
The source code for the project is available at https://github.com/linkedin/Cubert
Show projectAn Android application that allows a group of users in proximity to play music simultaneously in a synchronized fashion thereby creating a multi speaker system.
Designed and implemented the tiny FTP protocol using C (Multithreading with socket programming)
Gitbook is a recommender system where we created a novel algorithm to rank github projects using PageRank and Naïve Bayes Classification techniques. We (team of 3) crawled through all the github projects, and suggested recommendations based on the user’s profile which consist of area of interest (operating systems, machine learning, web development etc), preferred language (C#, Java, javascript, python ruby etc) and difficulty level of the user.