Recommendation 2: Help filers report correctly

There are many existing user interface patterns and best practices for form design that can help make FECFile more usable. In many of our sessions with filers, we found that the current design and presentation of many form fields causes confusion, which results in filers reporting incorrect or incomplete data. The FEC has made recent usability improvements to some aspects of e-filing with web-based versions of some forms, and we recommend that the FEC prioritize improvements to FECFile going forward.

A. Provide clear labels, suggestions, and hints

FECFile does not make it easy for users to identify which input fields are required, recommended, or not required. The resulting confusion generates a lot of calls to RAD, particularly around deadlines.

In some cases, filers are guessing as to whether they have input the right information or selected the right option from a dropdown.

“There was a drop down, but no place to write in text. We would have expenses that didn’t quite match what was in the dropdown. This led to questions like “we are trying to do this as accurately as possible, but if this category is not right, is the blame on us…?” I wanted to make sure I crossed Ts and dotted Is… oftentimes it didn’t seem clear how to list certain expenses from that dropdown menu.”

All form fields should be clearly labeled and accompanied by hints about desired data and necessary format. See figure 3 below for an example of a in-context help prototype that tested well during our research. In addition, each form field should have only a single meaning. You can find more of our recommendations on form field specifications in Recommendation 4. Finally, the user interface should clearly indicate which form fields are optional and which are required.

Figure 3. Example of a in-context help prototype developed as part of the e-filing study [view it in action ]

B. Remove redundant and unnecessary fields

When a filer creates a report, they enter information FECFile could use to customize subsequent forms and remove fields that are not needed for the filer to meet their specific reporting requirement. For example, when a filer opens the committee file, they must enter the committee ID before moving on to select the type of report they wish to work on. The types of forms a committee might be required to complete varies by the type of committee, but this is not reflected in the software, which currently presents all committees, regardless of committee type, with the complete list of all possible forms. This list could be tailored to show the filer only the types of reports that are relevant to their type of committee.

The FEC is already doing this in another area with their online web forms, which have been received favorably by the filing public and commercial software vendors. For example, the web-based version of Form 1 guides new committees through the registration process by asking them clear questions and then progressively tailoring subsequent options and form fields based on their answers. We recommend that FECFile be enhanced to take into account previously entered information.

In another example, a committee’s election cycle could be auto-populated based on what the system knows about the candidate, election, and state (which the filer entered when they created their report).

The system could also help users and prevent errors by allowing them to set certain fields as the default for a given work session. For example, when a filer creates a report, they enter the type of election and election year for the report, yet they are asked to enter this information again for each individual transaction. There are some cases in which a filer might want to modify this, but in most cases this information will be the same for all the transactions in the report.

“When you’re doing these reports, once you start one, you’re only doing it for that one [report]. Seems like it should automatically come up default to be the same one [election and election year] every time, and then you could change it if you needed to—It’d be a lot easier.”

C. Provide triggered and dynamic help

In addition to clearer form field display and labelling, FECFile should link to explanatory text about filing requirements. We heard from many filers that they often have questions throughout the filing process, including questions about campaign finance law, what information they need to report, and how to use FECFile. During research sessions, both experienced and novice FECFile users interrupted their reporting process to search for information about how to complete form fields. Providing contextual information within the FECFile interface will reduce the need for filers to switch back and forth between FECFile and external reference material.

We prototyped one idea for how to provide better context-specific information within FECFile by linking users from a specific form field to the glossary definition of data that belongs in that field.

“I think this is really helpful… having the definition right there, in the glossary, is extremely helpful. In the old software, there was nowhere to get that guidance—I would go to Google. But it was hard to match up, having the keywords line up and know what’s there to help you is great.”

D. Show users where they are in the process

Providing users with a way to track their progress through a complex process, such as e-filing, is a best practice in user experience design. Many users in our research cited time constraints as a major concern during the filing process. Providing clearer feedback about where people are in the process of submitting their reports could help them plan for the time they need and allow sufficient time for reviewing and correcting their information before deadlines.

In addition to notification of where they are in their reporting process, filers need clearer feedback that the system is processing their information, particularly when they are uploading their reports. Uploading can take a variable amount of time depending on the FEC’s server load, and without feedback that the upload is in process, filers sometimes quit or restart the upload again and again before the upload is complete.

A short story about the filer experience

For the data consumers who ultimately use FEC data, data quality is a primary concern. A 2014 study conducted by the Sunlight Foundation and the Center for Responsive Politics found widespread underreporting of required fields:

In the last six months, campaigns, PACs, individuals, parties and other organizations have, on average, submitted almost 5,000 blank required fields per month to the FEC, according to the FEC’s submission validation tool, which is distributed to vendors of filing software. And the most common errors are some of the most important to the voting public: 82% of required fields left blank have to do with identifying either the donor or recipient of a contribution.

The most common error is not providing basic, essential information like candidate information. Data validation could easily alert filers of these types of errors. Currently, data validation does not happen until the start of the submission process.

Giving users real-time feedback about their information can help reduce errors. We recommend improving how FECFile handles data validation by providing on-the-spot field validation for critical fields, promoting contact record validation, and making the process for resolving potential duplicate contacts easier.

Currently, FECFile performs minimal data validation: filers can submit forms through FECFile with required data missing or incomplete. The validation requirements were created at a time when online systems were new and people were uncomfortable with the concept of electronic filing. At that time, the FEC sought to minimize the impact of the change from paper to electronic reporting on filers, so the system’s design was based on existing paper forms, and the data requirements were flexible with few but the most egregious data errors being rejected. This system has since been outgrown by complex data needs of the Agency and those of the public.

Data users both at the FEC and outside have increasingly gone to automated data processing and review systems that do not function optimally with the loosely structured data provided by the current system.

When required data is missing, analysts in the Reports Analysis Division (RAD) may send a Request for Additional Information (RFAI).

As part of this study, we obtained a sample of the top issues that resulted in filers receiving RFAIs in 2016 (Table 1). RFAIs are sent when an FEC Campaign Finance Analyst needs additional clarification or identifies an error, omission or possibly prohibited activity.

Responding to RFAIs is a stressful and time-consuming process for the filer, because it often means they will need to amend their filing. Even small errors can compound, causing a cascade of amendments that take additional time and stress to untangle.

“Any error you ignore grows and becomes worse. It’s just something—you have to be constantly looking at it.”

"Sometimes I’ll spend hours just looking at it because as soon as I push that button, if I made a mistake, I’ll have to make an amendment. It’s not like 'if you made a mistake and you can fix it.' You’re going to get a letter."

In addition to costing filers time and stress, incomplete data also results in RAD analysts spending time on correspondence and phone conversations to help filers disclose information correctly, which may include guiding filers through fixing data issues. Better data validation could save analysts significant time. Analysts’ time is scarce; each RAD analyst is assigned to support between 250 and 400 Committees, and any time saved spent helping filers fix common errors could free up time to review reports or provide deeper support.

A. Help filers identify and resolve errors by providing real-time validation for form field entries

More feedback about data quality during the form completion process would improve the filer experience and would more evenly distribute the burden of data quality control among filers and the FEC. Transitioning control over data quality to filers upon data entry was also a key recommendation in the 2013 e-filing study.

The basic framework for tighter data validation is already in place; many of these errors already trigger warnings, but allow users to submit their reports anyway.

In-line validation should be implemented iteratively through a human-centered design and evaluation process. The risk of stricter data validation is that it can cause upfront frustration for filers. Right now filers can enter incorrect information without pausing, but in-line validation might require filers to address errors before moving on. Careful implementation can balance this risk with the downstream effects in a user-centric way, but more research is needed to know how validation should be implemented.

FECCheck already runs as part of the submission process for FECFile users, but FECCheck before submission is optional for vendors. To mitigate the effects of omissions and errors in the short term, the module that sends information from vendors to the FEC, FECLoad could be modified to run validation. The submission process could still allow the validation module to be called independently but also invoke validation as a part of the client-side upload process before submission.

Prototype testing of in-line validation

We prototyped one idea for in-line date validation in which filers received a warning when they entered a transaction date outside of the reporting period. Filers who tested this prototype largely missed this warning at the field level; filers, particularly power users, focus on the source of the information they are entering into FECFile (their spreadsheet or other account document), and they seldom look at the FECFile UI. This implies that while relying on real time warnings and validation may help users avoid misreporting, those alerts should find ways to signal the users as close in time to when the potential error happened, as soon as their focus returns to the FECFile UI. For example, using an audio indicator to indicate that an attempted submission of a transaction failed would draw the user’s attention back to the problem field shortly after it was entered (Figure 4).

Figure 4. Example of in-line date validation in a prototype developed for the filing study. [view it in action]

B. Help filers fix errors and omissions by providing clear warning and error messages

It is also equally important that the validation runs quickly and error messaging makes the problems clearly understood by the users so they can easily address data problems. Having more lightweight data checks through the process can also avoid filers having to deal with a large number of errors right before the submission. Without in-line data validation, the validator finds an error or warning just prior to submission. Because filers receive errors and warnings far from when they entered the data, they have trouble recalling and navigating back to where they need to make corrections.

Handling error and warning messages is further complicated by the fact that the messages that FECCheck returns are difficult to interpret and do not guide filers in how to handle the issue. For example, a user might get a warning that says “Leading Blanks {e.g. " TEXT"} not allowed,” which indicates that there are added spaces, or “blanks” that precede text they entered at some point earlier in their reporting. Since this could be referring to any number of form fields, figuring out first how, and then where to delete the added space can be like looking for a needle in a haystack.

Several interviewees mentioned error messages as a cause of frustration and confusion:

“When I was trying to submit and it wasn't going through it would be helpful instead of an error message, it told me where to go to fix it. In general more thorough information help.”

“The error messages are so confusing to people. Treasurer signature date? That’s the date you close the report, it doesn’t say anything about it in the software. The validation message should say “close your report”

Additionally, using more accessible language was one of the 2013 e-filing study recommendations.

Unclear error messages also create problems for vendors who use the FECCheck module, because these same confusing messages are returned via the API. This creates overhead for software vendors who either need to translate the messages in their own software or field calls from confused customers, as in the quote below.

“On the validator? [My] biggest complaint is explaining what the messages meant. We’ve had to translate that on our end [to be] sure I understood it. The validator isn’t wrong, it’s just cumbersome”
— Commercial software vendor

Some analysts in the Reports Analysis Division keep track of the errors and warning messages that frequently confuse filers. The FEC could leverage RAD expertise in creating improved warning and error message content.

C. Validate contact records as part of the submission process and make the reconciliation process easier

We recommend helping filers maintain clean contact lists by making contact validation part of the submission process, using validation logic that can catch more duplicate contacts (even when there are small differences in text), and improving the process for resolving duplicate contacts.

Filers often make small re-keying errors, which can result in duplicate contact records. These duplicates can lead to undetected excessive contributions or incorrect aggregate figures.

FECFile has a feature that checks for and helps resolve duplicate contact records when filers import their data, but we observed users struggling to move through workflow for checking and resolving duplicates. This used to be an automatically triggered feature, but the FEC was asked to turn it off because filers found the existing feature difficult to use. With this automatically triggered feature disabled, filers who enter data manually are never prompted to check for duplicate records or clean up their contact list. Some filers who chose manual data entry may not know that this feature exists at all.

We recommend that the FEC prioritize the iterative, human-centered design and development of FECFile’s capability for checking and resolving duplicates to make this feature more usable, and then make the new and improved feature a part of the submission process.

Prototype testing of finding and resolving duplicate contacts

We explored how filers might react to these changes by testing a prototype submission process that checked for duplicate contacts.

Filers reacted positively to the idea of making contact validation a part of the submission process, but several commented that they would need a way to bypass the process of resolving duplicate contacts if they were short on time.

“I like that this [merging duplicate contacts screen] comes up. I can very easily see multiple donors with similar info.”

“Seeing these every month may be annoying, but if you are doing the same report, there probably aren't that many. It may not be so bad to have because it will pick up fat fingering, and that is not a bad thing. Annoying, maybe, but I bet it catches more stuff that you’d be glossing over in reviewing the report.”

Filers appreciated when the prototype identified duplicates using “fuzzier” validation logic:

“This is a weird thing with FECFile: if it [the contact record] wasn’t pretty much exactly the same, it wouldn't pick up on the fact that those might be the same. I like that this is grabbing more things. Sometimes we get biz address and personal address and we can’t tell that it is the same person.”

Once potential duplicate contacts are found, filers need to be guided on how to resolve them. We tested several different designs for how filers might resolve duplicate contacts. The current system handles this by guiding users through merging duplicate contacts into a single record. We kept this same basic concept, but tested different UI designs for how this concept is communicated and visually represented.

We learned that the language and visual representation of what happens when you merge duplicate contacts needs to be crafted with care. In both our initial observations of filers using FECFile and in our prototype testing, users gave clear indications that they were confused as to what merging would do to the contact records in their database.

“Is this giving you the option to merge? Is it combining the transactions? What I don’t want to do is leave the record in there, that leaves you susceptible. You obviously can’t delete the info. Now how you get there, I don’t see. It’s not clear to me how the contributions and transactions are getting moved.”

Without a clear understanding of what merging contact records will do to their database, filers will avoid resolving duplicates.

The design of the merge feature needs to help filers quickly identify discrepancies between potential duplicate contacts by enabling them to compare like form fields directly adjacent to one another. We learned in our interviews and observations with filers that they determine whether a set of contacts are duplicates by comparing specific contact attributes rather than the contact records as a whole. Arranging potential duplicate contacts in rows with columns for the different attributes (name, address, occupation, employer, etc.) would support this comparison far better than the current design that places the complete contact records side by side (Figure 5).

Figure 5. Example of contact record blocks (top) compared to contact attributes arranged in rows for comparison (bottom). [Full size image]

A. Offering a web-based version of FECFile will allow filers to work the way they want to work

FECFile is currently a desktop application, which means it comes with limitations that constrain many filers’ workflow. FECFile is currently only supported on PC, but many filers want to be able to work on reports using multiple computers, including both Windows and Mac operating systems. Filers who primarily use Macs must find secondary computers, use a windows emulator, or seek out PC users who can file on their behalf. Filers also expressed a desire for cloud-based storage, access to FECFile away from the office, and the ability to collaborate with colleagues in other places to complete reporting tasks.

“He asked me to do the filing because he had a Mac and he couldn't use FECFile.” (interviewee has a PC)

“The one big drawback is that we use Macs in our office, and you can’t download it [the software] on a Mac.”

“I travel extensively and sometimes must complete the report from locations other than my office and computers other than my regular workstation. As a result, I must remember to download the data file to a USB drive and take with me to use. Then I must remember where my most recent data is stored. A web based system would eliminate this problem.”

“[a web-based system] would absolutely be helpful! There are two of us that work on the FEC filings and, currently, I have to wait until I can use the other person's laptop (where the FECFile software is stored) in order to enter the data needed for a filing.”

“[a web-based system] would be an outstanding upgrade. Many of my colleagues (and I) are Mac users and we've had to keep a PC around the office for the sole purpose of filing our monthly reports.”

“I live in constant terror of a computer crashed around filing time, even to the point of being afraid to make updates on my computer in the event that I may lose filing information that I have already entered. A cloud-based storage system would provide so much relief from those concerns.”

One way to deliver these capabilities is to move the software, which is stored locally on users’ computers, to a web application, and to store data on 3rd party servers or FEC servers, as it is entered into FECFile.

We were concerned that filers who opposed a web-based version of FECFile might be reluctant to be interviewed but wanted to give them an opportunity to share concerns anonymously. We conducted an conducted an anonymous web-based survey by sending the link to 8,906 unique email addresses. We received 533 responses. You can read more about our methods in the methods section. Our survey included 9 questions, seven of which were quantitative and two of which were qualitative.

Quantitative survey findings

The majority of respondents (74%) said they would use a web-based version of FECFile if it were available (Figure 6). Just 9% of respondents said they would not use it, and about 20% said they would need more information (Table 2).

Figure 6. Survey responses to the question of whether participants would use a web-based FECFile if it were available. [Full size image]

Table 1. Top issues that resulted in RFAIs in 2016

In general, the people who said they would use a web-based FECFile are already FECFile users, report small numbers of transactions, and are less likely to be compliance professionals. Of the people who said they primarily use FECfile, 84% said they would use the web-based FECFile if it were available. Those who said they would not use a web-based FECFile are more likely to use vendor software, be compliance professionals, and work for multiple committees, parties or PACs. This second group also submits reports with higher volumes of transactions.

Qualitative survey findings

We asked filers to tell us more about why they would, or would not, use a web-based FECFile and what questions they would have about it. We also asked participants to tell us their questions about a possible web-based version of FECFile; many responses were not questions but comments about people’s hopes and desires for the new system.

Of the 392 respondents who said they would use a web-based FECFile, 80 responded with substantive comments about why they would make that choice. 81% of respondents in this group expressed enthusiasm, and many mentioned specific capabilities that they are excited to see. The most common hope was that a web-based FECFile would let filers work remotely, followed by the hope that it would allow them to use a Mac computer. Many expressed excitement about the potential for collaboration on reports, cloud storage, or better access to the data. Several included stories about the current system crashing, resulting in lost data — along with hopes that a web-based system would prevent these losses in the future. A few respondents (6 people) had concerns with the security, speed, or performance of a web-based system, and one person (a current user of vendor software) expressed discomfort with the FEC having access to data before it is filed.

Of the respondents who would expect to use a web-based FECFile, 66 responded with substantive questions. 20% of comments evinced enthusiasm and impatience for a web-based FECFile, and 17% expressed desire for the new system to be more usable. About 11% of comments included questions about data security. About 10% of comments requested specific tweaks to the FECFile user experience. An equal number of comments (6%) asked that the new system make importing data easier or exporting data possible. A few respondents asked about accessing backed up data, whether the new system would still be free, whether users could use the old FECFile, and what would happen if the system crashes. Respondents also commented on how the new system might handle access control, after hours support, and improved instructions.

Of the 49 respondents who said they would not use a web-based FECFile, 22 responded with substantive comments about why they would make that choice. More than half (59%) of this group commented that they use commercial software instead of FECFile, so they do not care if it is web based. Five respondents would not be interested in a web-based FECFile because they only work on reporting when they are in their office. Three respondents commented that they prefer desktop software to web-based services because they believe they are superior in performance and reliability. In two cases, participants shared concerns about the FEC having their data on FEC servers before they would be ready to share it.

Only six participants who said they would not use a web-based FECFile responded with substantive questions. Two expressed concern that a web-based FECFile would infringe on commercial campaign finance software vendor’s market share. Other comments included concerns about the new system’s data security, quality, and performance.

Of the 92 respondents who said they would need to know more before deciding to use a web-based FECFile, 56 responded with substantive comments. About one-third of the comments from this group of respondents said that they would need to know more about how data is secured and accessed before they could decide whether they would use a web-based FECFile. We intentionally provided very little information about how the system might be implemented, because we wanted to learn what concerns people had that the FEC would need to address in the system’s design. 12% of comments raised questions about how the new system would integrate with their current way of doing things, and how they would access their data stored in the current FECFile. An equal number of comments also expressed questions about whether they could continue to be able to use the desktop version of FECFile. A handful of respondents echoed comments made by the “yes” group regarding capabilities they hope to see; specifically, an easy-to-use import function, the ability to set access controls, and an overall better user experience. Three respondents expressed fear that making FECFile web-based would negatively impact the speed and reliability of the system; others felt more comfortable with a web-based system that could back up their data to the cloud.

Thirty one of the respondents who would need more information before deciding to use FECFile responded to our survey with substantive comments about their questions. Two themes appeared in 23% of comments each: concerns about data security and a desire for the new system to make it easier for filers to import their data. About 13% of comments questioned whether the new system would come with improved help and/or support. A small number had questions about whether they could choose how to file reports going forward. Remaining questions were about exporting data from FECFile and how data would be transferred from FECFile to the new system.

Finally, we asked users to tell us which of the following capabilities they were the most excited about: access to reports in progress from multiple devices; data backed up on central servers; Mac compatibility; data backed up in the cloud; and real time collaboration on reports. Their answers are shown in Figure 7.

Figure 7. Survey responses to the question "Of the following capabilities we hope this change will provide, which are you most excited about?" [Full size image]

B. Simplify and promote importing to reduce transcription errors

Most information in FEC filings comes from other tools or processes: financial and accounting software applications (such as Quickbooks), payroll application exports, bank statements, membership databases, or physical check registers. Filers using FECFile have two options for inputting this financial data: they can transform and import it or manually enter the data by hand.

Most users we spoke to expressed a desire to import their data but many did not know it was possible. (Those filers who do use the import feature are often more experienced.) This lack of awareness may be a result of the current FECFile user interface, which does not make it clear that importing data is an option.

For those who do know about the feature, preparing data for import requires substantial pre-processing to get it into a format FECFile will accept. Even experienced filers may avoid using it for small numbers of transactions. Even when imported, some data includes a manual component: data with parent-child relationships such as memo entries must be entered manually. For this reason, many experienced filers said they choose to enter all disbursements manually, even when it means inputting a relatively high volume of transactions one by one.

We designed a prototype to test our hypothesis that improving awareness and usability of the import feature would increase adoption of data importing, which in turn reduces the creation of duplicate records and manual data entry errors.

For our prototype, we required users to choose how to enter their data: on starting the process, users were greeted by a decision to enter manually or to import. After this, we observed the users for errors (wrong clicks) and reactions in attempting to proceed through the import flow. These tests revealed that by placing the choice of import or manual entry directly in line with the data entry flow, users were able to initiate the import process, and that once in the flow, users were able to understand what was happening and complete the import with little difficulty: how to choose a file to import, how to choose which columns in their spreadsheet map to data elements in FECFile, initiate the import, preview the results, and confirm the success of the import. Many users specifically called out their preference toward the idea of mapping columns.

“A lot of our current work is formatting the Excel file. If were able to select from the import file that we have, that would be really great. I would like to be able to select the columns that I need from the spreadsheet.”

“That’s nice… So I guess if those [referring to a import file column] weren't names I could say ‘oh no this is the employer or whatever.’ I like that very much. So if someone didn't have the spreadsheet in the right order, it would make an attempt to understand what they’ve got. Now they give you a dummy sheet but they talk about it in such a convoluted way, it takes a phone call to RAD to understand.”

“My friend referred me. He opened a PAC for the Bernie Sanders campaign. He was selling t-shirts he was making to raise money for the campaign. I didn’t know importing was an option. That probably would have saved me a lot of time.”

Figure 8. Example of import feature prototyped as part of the e-filing study [view it in action]

C. Provide an export capability

FECFile does not currently offer a way for users to export data into a spreadsheet form, which hinders the way they can manipulate data and their ability to transition between different systems and software. Adding this capability would help many users groups including filers who use FECfile and commercial filing software, the FEC Audit Division, and commercial software vendors.

Filers can sometimes lose their reports (in the case of a natural disaster or where their computer is lost or damaged) and without the ability to export from FECFile, they have to “rebuild” their entire database. Were the data exportable, filers could simply import it back into FECfile, which would prevent rekeying errors and save them time. In addition, filers could use an export feature to sort and manipulate their data in a spreadsheet, which would help them find and resolve errors and amend reports.

The need to manipulate report data in spreadsheet form is also shared by the FEC’s Audit Division, and auditors have asked for an export capability in the past.

An export capability would help vendors easily transition FECFile users to their software by enabling them to download data from FECFile and upload it to the vendor’s system, rather than rekeying it.

Finally, adding an export capability is a milestone on the way toward transitioning users from FECFile to a web-based system, as the FEC will need to have a plan in place for getting data out of FECFile and into the new system.