SkinnerDB: Regret Bounded Query Evaluation using RL

Channel:
Microsoft Research
Subscribers:
344,000
Published on ● Video Link: https://www.youtube.com/watch?v=Qy-JdTYe1u8


Duration: 1:21:04
932 views
23

Query plans are typically selected based on coarse-grained data statistics and simplifying cost and cardinality models. This approach is brittle and the most common cause of highly suboptimal performance in practice. SkinnerDB eschews statistics and models in favor of "intra-query learning": a principled "learning by doing" approach, based on reinforcement learning.

SkinnerDB does not learn from past queries to optimize the next (inter-query learning). Instead, it divides the execution of one single query into hundreds of thousands of micro-episodes. By trying different plans in different episodes, SkinnerDB learns optimal, query-specific plans from scratch. By selecting plans based on sound statistical principles, balancing exploration and exploitation, SkinnerDB guarantees near-optimal expected execution time. Execution ends once result fragments can be combined into a complete result.
Intra-query learning requires a specialized execution engine. We need to facilitate fast plan switches, avoid plan-specific intermediate results, and obtain unbiased quality estimates after executing plans only for a few steps. In this talk, I will introduce the basic principles behind intra-query learning, discuss SkinnerDB and our specialized execution engine, and present experimental results for our latest prototype.

See more at https://www.microsoft.com/en-us/research/video/skinnerdb-regret-bounded-query-evaluation-using-rl/



Other Videos By Microsoft Research


2020-09-03Creating Animations in Microsoft Expressive Pixels (4 of 9)
2020-09-03Creating an Image in Microsoft Expressive Pixels (3 of 9)
2020-09-03Getting Started with Microsoft Expressive Pixels (2 of 9)
2020-09-03Welcome to Microsoft Expressive Pixels (1 of 9)
2020-09-03Importing Animations in Microsoft Expressive Pixels (9 of 9)
2020-09-03Galactic Bell Star Music Demo
2020-09-03Directional Sources & Listeners in Interactive Sound Propagation using Reciprocal Wave Field Coding
2020-09-01Platform for Situated Intelligence Overview
2020-08-28Programming with Proofs for High-assurance Software
2020-08-14From SqueezeNet to SqueezeBERT: Developing Efficient Deep Neural Networks
2020-08-12SkinnerDB: Regret Bounded Query Evaluation using RL
2020-08-07From Paper to Product
2020-08-06Can we make better software by using ML and AI techniques? With Chandra Maddila and Chetan Bansal
2020-08-05MineRL Competition 2020
2020-08-04Directions in ML: Algorithmic foundations of neural architecture search
2020-08-03Microsoft Urban Futures Summer Workshop | Policy and Social Impact [Day 3]
2020-08-03Microsoft Urban Futures Summer Workshop | Sensors and Data [Day 2]
2020-08-03Microsoft Urban Futures Summer Workshop | Data Driven Urban Transformation [Day 1]
2020-07-31Managing Tasks Across the Work-Life Boundary: Opportunities, Challenges, and Directions
2020-07-31Phong Surface: Efficient 3D Model Fitting using Lifted Optimization
2020-07-30How Work From Home Affects Collaboration: Information Workers in a Natural Experiment During COVID19


Tags:
SkinnerDB
reinforcement learning
intra-query learning
Immanuel Trummer
Microsoft Research