Thor Whalen

Ex-Mathematician, Ex-CTO of OtoSense, sound recognition start-up in the silicon valley, sold to Analog Devices. Now works as a director of AI research at Analog Devices.

Talking to data as you would with built-in types.

ML, AI, & Data, Intermediate
8/18/2019 | 3:20 PM-3:50 PM | Mya

Description

In this talk I will present a project (coined py2store) whose goal is to develop tools that allow developers to interact with a variety of data sources and sinks, local or remote, through a consistent and simple interface.

How simple? The simplest. Objects that feel like built-ins (dicts, lists, sets).

Abstract

Py2Store might make you think about ORMs. If so, think again. The intent of py2store is not to map objects to db entries, but rather to offer a consistent interface for basic storage operations. In that sense, py2store is more akin to an implementation of the data access object (DAO) pattern. Of course, the difference between ORM and DAO can be blurry, so all this should be taken with a grain of salt.

How many times did someone share some data with you in the form of a zip of some nested folders whose structure and naming choices are fascinatingly obscure? And how much time do you then spend to write code to interface with the freakish structure?

One of the intents of py2store is to make that easier to do. You still need to understand the structure of the data store and how to deserialize these datas into python objects you can manipulate. But with the proper tool, you shouldn't have to do much more than that.

You have a new project or need to write a new app. You'll need to store stuff and read stuff back. Stuff: Different kinds of resources that your app will need to function. Some people enjoy thinking of how to optimize that aspect. I don't. I'll leave it to the experts to do so when the time comes. Often though, the time is later, if ever. Few proof of concepts and MVPs ever make it to prod.

So instead, I'd like to just get on with the business logic and write my program. So what I need is an easy way to get some minimal storage functionality. But when the time comes to optimize, I shouldn't have to change my code, but instead just change the way my DAO does things.