In order to monitor the engagement, satisfaction and willingness to promote the business of the company, a multinational telecommunications corporation conducts a survey taken by 200,000 employees three times a year. Because the survey provides an opportunity for employees to make free-form comments, the client needed a mechanism to identify and report on comments that implicate “mandatory subjects of bargaining” as set forth under the National Labor Relations Act, and/or subjects that were negotiated in the applicable Collective Bargaining Agreement (CBA), as there are limitations on management’s access to and use of any such comments, as well as formal reporting requirements.
Additionally, the client wanted a way to generally categorize and track comments raising certain other business issues and/or concerns. Finally, certain survey comments, regardless of whether they implicated mandatory subjects of bargaining, may raise sensitive potential legal issues (e.g. discrimination, harassment, retaliation) that may require further investigation and escalation.
The survey is conducted by an independent vendor over the span of a two to three week period. Once review is completed, the survey vendor needs the final categorization tagging incorporated with the related original data, and returned in raw data form for further processing and reporting. Additionally, the client needs the final data set within days of the closing of the survey.
Counsel On Call technology and process experts, working with a review hosting technology vendor, devised a process by which the comments are converted into individual text documents that are loaded into an established document review and coding platform. This process allows COC to efficiently batch and manage the review of comments by a team of approximately 15 attorneys, who efficiently review and tag the comments for CBA, non-CBA and legal issues with a defensible quality control process. Additionally, the process was designed to allow COC to utilize searches and highlighting of key terms likely to implicate CBA and other legal issues, which increases both the speed and accuracy of review.
Once the initial review is completed, certain core team members perform various quality control functions to ensure accuracy and consistency in coding throughout the data set. Once this is completed, the technology vendor compiles the COC work product, along with the original data, and returns it in a raw data form to the survey vendor for further processing and analysis. Additionally, COC provides, directly to the client’s legal team, a final report that includes all comments raising legal issues, both in full list format and broken down by legal issue.
Counsel On Call has conducted this quarterly compliance survey for several years. Typically, there are 140,000 independent survey comments to review. In order to meet deadlines, COC receives multiple data pulls while the survey is pending; final review and quality control is always completed within the targeted deadline and, on average, takes six to nine days.
Counsel On Call has a core team of attorneys who work on this review every quarter, which improves knowledge retention, consistency, accuracy and efficiency. Quarterly trainings are hosted. In all, the team, technology and process developed allows the team to reach average review speeds up to 250 comments per hour.
Over time and with thorough streamlining of the process, the cost is down to $0.37 per comment.