Optimizing Stream Ingestion & Task Monitoring with Rust and React Admin - Episode 208
8In this video, I take you through a deep dive into debugging and optimizing the stream ingestion process while working on task monitoring features for the 'Glowing Telegram' project. Utilizing Rust for back-end development and React Admin for the front-end, I demonstrate effective problem-solving and strategic improvements to our workflow.
We kick things off by addressing issues related to the step function, which was previously overloaded with transcription segments. By saving data directly in DynamoDB and fetching it on demand, we minimized data passing through the step function, boosting its efficiency. After refactoring some messy code and improving function clarity, I ran builds, deployed updates using the CDK, and implemented changes to streamline deployments via GitHub Actions in the future.
As part of enhancing usability, I explored ways to automate deployments for back-end changes—improving the process for pushing front-end updates linked to CloudFront while maintaining versioned deployment options. I also emphasized better task monitoring, ensuring task statuses like uploads and ingestion are accurately presented in the UI.
Later in the stream, we tackled trouble with event bridge integrations and schema generation. I shared strategies for aligning JSON schemas between Rust and TypeScript for seamless back-end-to-front-end communication. Along the way, we encountered typical software development hurdles, like dealing with caching issues and debugging DynamoDB table entries for task records—turning abstract obstacles into teachable moments.
This session highlights the iterative nature of debugging and refining a coding project, touching on multi-language workflows, collaboration with tools like GitHub Actions, and the importance of strategic planning when working with dynamic systems.
đź”— Check out my Twitch channel for more streams: https://www.twitch.tv/saebyn
GitHub: https://github.com/saebyn
Discord: https://discord.gg/N7xfy7PyHs