Disclosure Data Weblog

Posted by: Bob in Campaigns
Published: 08:00AM Dec 30, 2009

As promised, we've added a 2010 candidate summary file to the list at data.fec.gov. This file contains a record for each candidate who has registered with us or appears on a state ballot for a 2009 or 2010 Congressional race.

For those of you who have been using data from our ftp server over the years, this file is analagous to the "webl.zip" files - the same rules for including candidates and calculating totals. The big differences here are that we've included ALL of the information reported by campaigns on the summary and detailed summary pages of their filings in this file, where webl only included a subset of this information. So, for example, the candidate summary includes the total received by the campaign in contributions from individuals where the specific contributions sum to less than $200 per person so the specific information doesn't have to be included in the filing. This will hopefully help people get a sense of the full breakdown of contributions by size.

Check out the "customize data" box - it allows you to isolate just candidates in a certain state or district or just one party or just challengers or open seat candidates, among other options. As always, you can sort the results however you choose and/or download the data in different formats.

The data in this file matches what you see when you use the 2010 campaign finance map and adds (we hope) another level of comprehensiveness and flexibility to our presentation of campaign finance information.

Let us know what you think.

Permalink | Comments[29]

Comments:

Bob, Will this file be updated daily like the FTP candidate summary file is?

Posted by Derek Willis on December 30, 2009 at 10:39 PM EST #

Derek, The update schedule for these files is the same as for the older data on our FTP server - generally by noon eastern time each day representing changes completed the previous day. We're looking at that schedule, though, and may make some changes to allow the data to appear more quickly. We'll let everyone know if and when that happens

Posted by Bob on January 04, 2010 at 11:26 AM EST #

This is great, Bob. Do you have a data description for all these fields that you are now providing available? Susi

Posted by Susi Alger on January 21, 2010 at 06:20 PM EST #

Susi, The descriptions of the new data fields are in the metadata page for the file - http://www.fec.gov/finance/disclosure/metadata/metadataforcandidatesummary.shtml Let us know if you need more.

Posted by Bob Biersack on January 22, 2010 at 08:54 AM EST #

Look at the following line using a courier font:

line 1158 from CandidateSummary.csv file today: "H0SC05031","MULVANEY, JOHN MICHAEL "MICK''","H"

The "Mick" nickname above is preceded with a single doublequote and followed by two single quotes.

Can you fix problems like this where the text qualifier is used within the field?

While Excel can parse this, I'm trying to use R to analyze the data and R chokes on parsing this line.

Thanks for listening.

Posted by Earl F Glynn on February 02, 2010 at 11:01 PM EST #

So what does it mean with when the Candidate Summary file is missing a candidate? Jordan for Congress (KS-3) filed: http://query.nictusa.com/cgi-bin/dcdev/forms/C00437996/450761/ But I can't find Nick Jordan in your Candidate Summary? http://www.fec.gov/data/CandidateSummary.do?format=html That summary shows a Harold, Jack and James Jordans but no Nick?

Posted by Earl F Glynn on February 09, 2010 at 12:30 AM EST #

Earl, It looks like Mr. Jordan didn't specify a year of election when he filed his statement of candidacy a couple of weeks ago. We should be able to take care of that problem so that his summary information will appear in the candidate summary file. I would look for it next week (we're still a little snow-bound here. . .) thanks

Posted by bob on February 11, 2010 at 01:17 PM EST #

Hi. Is there a way to see this data but only for those who have filed a statement of candidacy?

Posted by Matt on February 19, 2010 at 11:33 AM EST #

Is it possible to provide machine consumable metadata for reports like this. Providing XML Schema (XSD) for XML files would enhanse the consumability of your data. Thanks ~john

Posted by John Doyle on April 20, 2010 at 03:27 PM EDT #

The problem reported back on Feb 2 is still a problem with the current data. Having quotes inside a quoted field with the CSV format can cause a problem. Excel seems smart enough to parse the line with the following, but the R statistical language is not. "H0SC05031","MULVANEY, JOHN MICHAEL "MICK''" [view above with courier font]

Posted by Earl F Glynn on April 21, 2010 at 01:19 AM EDT #

Earl, We've changed the name for the candidate you identified here, and we're working through what look to be 25 or so others where quotation marks appear in the name. Sorry for the delay, let us know if you continue to have problems.

Posted by Bob Biersack on April 22, 2010 at 02:14 PM EDT #

Bob: Thanks for fixing the data. R reads the latest CSV file now. [I can read the XML version too but I'm still learning how to parse XML in R -- that's not just one line in R like read.csv is.] I'm creating some statistical reports using R to verify correctness of data as a quality control after reading a new dataset. Do I report other problems with the data here, or is somewhere else better? For example: line 282 in the file I downloaded today is for Harley Delano Brown in Idaho (can_id= H0ID01170). The data in the can_off_dis (candidate district) and can_par_aff (political party) fields seem to be switched for this candidate. Now can_off_dis="RE" and can_par_aff="1". But, I found the guy's web site and it appears these two fields for him should be can_off_dis="1" and can_par_aff="REP". Is any sort of data validation done when data are entered?

Posted by Earl F Glynn on April 23, 2010 at 10:28 PM EDT #

Why did I spend a lot of time to post a comment about invalid districts only to get a message that it was marked as spam? Please post an E-mail address for E-mail discussions of problem data. Your comment was marked as spam and will not be displayed. ¿Comment has more than 1000 characters

Posted by Earl F Glynn on April 24, 2010 at 01:37 AM EDT #

Bob, Glad to hear that you're working on the double-quoted nicknames. We had some problems parsing those fields as well. Came across this example in the summary file today: AHERN, ELEANOR C "ELLIE"

Posted by Troy Thibodeaux on April 24, 2010 at 07:01 PM EDT #

Earl, Sorry about the spam limit - its a feature of the blog software we're using - gets me too. (I do see the message even when its marked as spam.) Email me directly (bbiersack@fec.gov) if you have lots of information to send. State, district and party information come to us from two sources - the statement of candidacy filed by each candidate, and the official ballot list sent to us by the state elections official. Correcting these data from other sources is tricky for use because the data we provide should generally come from official filings. Its not an absolute rule so we fix these as we (you) find them. Thanks for being so thorough and careful in your work.

Posted by Bob Biersack on April 26, 2010 at 10:12 AM EDT #

Bob, Is there any way to download individual contribution information? I.e., to get, say, Barbara Boxer's list of individual contributors without going to each separate alphabetical page and copying/pasting? Thanks! Danielle

Posted by Danielle on June 08, 2010 at 02:21 PM EDT #

Danielle, Right now, there's a way to download individual contributions but its pretty ugly. There is a file on our FTP server - http://www.fec.gov/finance/disclosure/ftpdet.shtml#a2009_2010 that contains any contribution from an individual where the amount was at least $200 during 2009 and 2010. (You can get the equivalent file for earlier cycles too.) The format is archaic and hard to work with, though, and you'll need to combine it with at least one other file (committee master) to get all of the information you need. We're working on something similar to our candidate disbursements files for their receipts as well, but we're not quite there yet. If you need more information feel free to send me an email. Thanks

Posted by Bob Biersack on June 09, 2010 at 01:05 PM EDT #

The Candidate Summary XML file contains six political party abbreviations that are not in the Candidate Master File's list of party designations. They are CON, NPA, FED, GOP, JCN, and GRN. While I might be able to guess what some of these are, is there a more complete list somewhere that tells what parties these stand for? Thanks.

Posted by Drew on June 23, 2010 at 02:32 PM EDT #

Drew, This comes (partly) from using an open ended question to get party information when candidates register with us. We'll add these to the list, but in the meantime here are the ones you've identified; CON = Constitution FED = Federalist GOP = Republican GRN = Green JCN = Jewish-Christian National NPA = No Party Affiliation (Florida) thanks for pointing these out.

Posted by Bob Biersack on June 23, 2010 at 03:12 PM EDT #

Bob, I wish the UK were as open as this with all their accounts, I am doing a project at night school to show the difference in USA and UK candidate for office. Getting information on this side of the water is proving difficult.

Posted by Bill the courier service man on July 27, 2010 at 03:33 PM EDT #

Is this same information availble for 2008? If so, can you provide a link?

Posted by Barrett on August 05, 2010 at 01:24 PM EDT #

Is there a way to tell if the candidate is in the general election post primary?

Posted by Bob waterman on September 19, 2010 at 07:20 AM EDT #

I have the same question as Barrett--is this data accessible in this format for previous election cycles?

Posted by Austin on October 06, 2010 at 12:19 AM EDT #

I hope UK, India and also other leading countries were also open like USA. Is there any option present to get the information for the past elections.

Posted by George on October 19, 2010 at 01:01 PM EDT #

Your right George, I wish my country is just as open like the US

Posted by Steve Robbins on November 06, 2010 at 02:40 AM EDT #

Hi folks, It seems that your 2012 candidate summary files are missing the House/Senate filings for at least some of the presidential candidates. While McCotter has a presidential and House filing in the data, neither Ron Paul nor Michelle Bachmann have their House filing listed. Is this something that you can remedy? Thanks again for the great resource! Susi

Posted by Susi Alger on October 19, 2011 at 02:48 PM EDT #

For some reason, some of the House candidates did not get a final update for the 9/30 period. They show up on the http://www.fec.gov/disclosurehs/HSCandList.do page, but show a closing of 6/30/11 in the Candidate Summary csv. Ideas? Scott

Posted by Scott Lay on October 31, 2011 at 02:09 PM EDT #

Susi, As we discussed, our intention is to only provide one filing for each candidate the presence of both McCotter's presidential and house filings in recent downloads is a transitional phase as his presidential campaign winds down. Paul is not running for the House in 2012 and Bachmann has re-designated her old House PCC as her Presidential other authorized committee. The Committee Summary File will allow you to look at Bachmann's, Paul's and McCotter's individual committees. The committee summary does not sum financial activity by candidate.

Posted by Paul on November 10, 2011 at 10:40 AM EST #

Scott, I'd need to know the specific candidate or district list of candidates to answer your question more completely. There are a number of reasons the through date not be September 30, 2011. One may be the committee has terminated. Currently, there are 55 candidate committees on the 2012 House and Senate Map that do not have a through date of September 30, 2011.

Posted by Paul on November 10, 2011 at 10:49 AM EST #

Disclosure Data Weblog

Search this Blog

Related Links

Categories

Feeds

Most Popular Tags