Reposted with permission from AALL Spectrum, Volume 26, Number 2 (November/December 2021), pgs 42-43.
By Diana Koppang, Director of Research & Competitive Intelligence, Neal, Gerber & Eisenberg LLP
To continue to lead, librarians must build on their existing expertise by gaining data science fluency and proficiency with new data-driven tools.
In the 2021 AALL State of the Profession report, 52 percent of private law library respondents stated that they did not have an AI/Machine Learning Initiative and had no plans to start one. I may have been among those 52 percent (honestly, I can’t remember that far back). If so, then I too fell into the common habit of downplaying my technical expertise as a librarian. We must stop doing that.
Law librarians have been among the lead users of artificial intelligence (AI) and machine learning technology in law firms since the advent of this technology in law ‑rms. Early machine learning in legal tech appeared in legal research platforms and e-discovery software. It’s only recently been expanding into the fields of process optimization, contract review clause analytics, and other knowledge management solutions. So, because librarians are often not part of those new initiatives (even though we likely should be) we think we are not promoting advanced technology within our organizations. But we have been promoting it—and at times necessarily pointing out the flaws in developing tech.
I think the survey question was perhaps misleading (though not intentionally). I would have read this as “new” initiatives. I would have also only answered in the affirmative if the library was leading the initiative rather than being part of a cross-departmental team that was adopting new tools that utilized AI and machine learning. But this likely goes back to librarians’ tendencies to downplay their achievements and expertise. I consider myself pretty good at promoting my field and my department, but it’s still something I must constantly work at.
The research platforms we use every day are frequently enhanced through machine learning. The “big box” search tools in our research platforms utilize machine learning to continuously improve the relevancy of the results. So, then why not say attorneys who use the same platforms are also experts at AI and machine learning? Because they are not the ones who are regularly assessing that these tools live up to the hype and are worth the cost.
When the big legal research platforms promoted their big box searching, which produced insane volumes of results—many of which were not relevant—it was the law librarians who did the work to point out these issues. We also pointed out that more results are not necessarily better than relevant results. As Neil Gaiman infamously stated, “Google will bring you back 100,000 answers. A librarian will bring you back the right one.”
A quick aside on that quote: While I’m sure we all appreciate the compliment; we can’t stop at being the experts who know how to find the right answer. We have a professional responsibility to provide feedback and guidance to these platforms to help them be better at bringing back the “right” answer. And of course, we have a responsibility to teach information literacy and research skills to empower others to find the right answer.
Law librarians find the flaws; we make recommendations for improvement. We do the painstakingly slow work of comparing results to raw data to assess the accuracy of these tools. Good law librarians never take the vendors’ word that a flashy new toy does what it purports to do. We seek proof and often we must do that work ourselves when the vendors are reluctant to even crack the lid of the black box of their technology. Ultimately, hopefully, the law librarians are the ones who make the call as to whether the new tech adds enough value to be worth the financial cost to our organizations. All this work takes basic understanding of data quality, categorization, and optimization. Most of us have this understanding, even if we have never stopped to realize it.
As an example, as the field of litigation analytics developed rapidly, resulting in exciting but costly tools, it was law librarians who stepped up to analyze the analytics. We tested the platforms against raw docket data to highlight the nuances of these platforms that produced such varying results. is led to the platform developers making needed changes to the platforms or clarifications in their scope notes.
So, if I have convinced you that we are already data scientists of a sort and out of necessity, I would argue that we need to continue to build on these skills.
First, we need to get better at understanding the vocabulary of data science and basic concepts. We are already halfway there because of our training in cataloging, including the concepts of name normalization, data accuracy, and information organization. If we can’t speak the language of data science, it will be hard for us to be taken seriously in this area. A colleague refers to this as “tech fluency,” and it is a necessary twenty-first century law librarian skill.
Second, we need to gain the technical skills to display and explain the data we know how to find and organize, such as proficiency with data visualization tools such as Power BI, Tableau, and DataHero. Everyone does not need to be a master builder, but basic ability should be required right alongside other functional skills such as Microsoft Excel and Microsoft Word.
Third, we need to more fully understand the data that powers our organizations and how to map the connections or identify the lack of connections. The future of legal tech is undergirded by data. New tech will seek to leverage that data. But as our profession well knows, bad data will never be fixed by new tech. Bad data in equals bad data out.
Lastly, we need to continue to lead in educating our patrons and stakeholders on the value of good data and what can be developed through managing and leveraging both internal and external data. We need to open the black box of technology tools that intimidate attorneys and other patrons. We can never stop being teachers.
The need for data scientists in law firms will only continue to grow. By building on the experience we already have, law librarians are a natural fit for evolving into those roles.
On Firmer Ground 