Disclosure Data Weblog

We're ready to start providing files in the data catalog that contain detailed information about the specific receipts and disbursements for candidates and committees.  This will include, for example, details for all contributions from individual people where the aggregate amount the person has given to a committee exceeds $200.  Similarly, all payments by committees to specific vendors will be available once those payments have exceeded $200 to that vendor.  Obviously, these files will contain lots of data - potentially millions of rows.

One problem we're having, therefore, is deciding what groupings of candidates or other committees we should use as the starting point for these files. (We're also keeping in mind the need to search these data based on information about the donor or the entity being paid - if you have ideas about how these might be grouped in more manageable sets of information, let us know.) We've got lots of ideas and we'd like to know what you think. Use the comments section to tell us your thoughts on these or other ways of organizing the information that would be helpful for you. 

We're beginning with data from the 2009-2010 time period, and when we've settled on a process we'll expand with more historical information.

First, we're thinking about placing the largest sets of itemized data in XML (if its not too big) and CSV files on our FTP server so if you choose something like "all 2010 candidate receipts" from a listing in the catalog you would be redirected to a zip file in the format you choose.  These would be updated once a day.  Is there a different file format we should consider because these files are so large?

We might do this for a number of groups - e.g.:

• All 2010 Senate campaign receipts
• All 2010 Senate campaign disbursements
• All 2010 campaign receipts in [STATE}
• All 2010 campaign disbursements in [STATE]
• All receipts for PACs
• All disbursements for PACs
• All receipts for Democratic national party committees
• All receipts for Democratic state and local party committees
• Etc.

Do these look like the right groupings to work with? Are there others that would be helpful to you?

No matter what, we'll offer some "customize" options that will allow for more specific requests - and we're working on a process that would allow you to choose a specific candidate or committee and get a package of two files - one for receipts and one for disbursements, with just one click. Is there anything else that would be useful?

Thanks

Permalink  |  Comments[13]