This is an old revision of the document!
Cumulizer
This is a Make OpenData.ch Hackday Bern 2013 Finance-themed project with a focus on analyzing personal shopping data. We have started by aggregating data available from the Cumulus purchase points program run by Migros, but would be interested in expanding the concept to Coop Superpunkt and others.
Fundamentally, we believe that the data we help to collect when we make purchases is relevant and useful to us as individual shoppers, and we want to try provoking popular action to aggregate this (anonymously) with others in our community. We want to learn some of what the shops are already learning about us - what, when and where we make our purchases. Since their data is not open, we need to liberate our personal data by uploading it to a shared repository.
Live site: http://www.cumulizer.eu
<GITHUB cstuder/cumulizer>
Data
Right now any customer who takes part in the Cumulus program can access detailed data about their purchases in CSV format. This is an excellent initiative from Migros which is somewhat hampered by a not very user friendly web application - we want to automate this process to make it easier for users.
Right now here are the steps to collect your personal data:
- Log into the Cumulus program using your customer number and a password on a paper mail-out to create an M-Connect account if you have not already
- Under Mein Konto - Kassenbons you can browse and view details of your shopping receipts
- Browsing month-by-month, and for every page, you need to click Alle, and then Ausgewählte Kassenbons als Excel-Liste (csv) (not “Übersicht”) to download a file with the details for those shopping trips
You are now ready to share your personal data with us. At the moment we do not have a live application, but if you are really eager to help, please send us your CSV files directly to cstuder AT existenz DOT ch
Legal status
Here is what Migros has to say about data usage in their data protection policy:
Das Datenschutzgesetz bestimmt in Artikel 8, dass jede Person von Inhabern einer Datensammlung Auskunft über die zu ihrer Person gespeicherten und bearbeiteten Daten verlangen kann. Diese Auskunftsbegehren müssen schriftlich eingereicht werden. Auf der Homepage des eidgenössischen Datenschutz- und Öffentlichkeitsbeauftragten finden Sie entsprechende Musterschreiben.
(rough translation: they link to the government standard form which you can fill out to request an export of all your personal data)
In the Terms and Conditions and Impressum there is no mention of restrictions on the use of the personal Cumulus data. At the moment we assume, and will contact their management to check, that they will not restrict us from using the receipt data from our personal accounts any way we can, including sharing it publicly. We will obviously not try to collect any other person's data without their full cooperation, attempt to circumvent any of their security, or put any automated scraping/spidering in place. We are really hoping to find a way to cooperate with the Cumulus program, not abuse the allowances they have made that make this project possible.
Project status
We are working on the initial prototype, aiming to complete a basic Phase I by tomorrow and give a view into the user's own data.
In Phase II we are aiming to aggregate data from the community, allowing multiple users to share their data anonymously, and show common trends.
In Phase III we will be able to link our data with other sources, showing additional information about users purchases, and how buying trends align with, for example semantically interlinked economic data (world bank via 270a) and open consumer surveys (data.gov).
Example
Development information
Link | Info |
---|---|
/dashboard/simpleupload | Simple upload form |
/dashboard/heatmap | Heatmap of the stores |
/maintenance/geocodestores | Geocode store adresses |
/maintenance/autocategorize | Start autocategorization |
/api?action=summary | General statistics |
/api?action=stores | Stores and sales |
/api?action=monthlypurchases | List of purchases by month |
/api?action=spendings | Monthly spending sums by category |
/api?action=categories | List of all categories |
Install information
- Download Cumulizer from GitHub and upload to your site
- Create a database (DB) for Cumulizer
- Modify “USE cumulizer” with your DB name (_docs/cumulizer.sql)
- Modify username and password (application/config/database.php)
- Upload receipts (dashboard/simpleupload)
- Run geotagging generator(admin/geocodestores)
- View the heatmap
(dashboard/heatmap)
Team
Links
- Solikarte is a related initiative for anonymously collecting points for charitable causes, and would be a possible data source and partner for this project
- One Receipt lets customers in the US see all their purchases in one place
- Opening Product Data for a more Responsible World (OKFN Blog) talks about the opportunities for open product data, and the Product Open Data project
- A Deep Dive into Facebook and Datalogix (Electronic Frontier Foundation) explores the marketing bonanza that is consumer data collection, and warns of the privacy risks
- What does the consumer data industry know about you (The Atlantic) explores a few more perspectives on the subject