This article was originally written by E. Costa and sent to the members of the Della Vos group. The text below is a translation from the original in Portuguese, with some minor editorial additions.

There are two ways to investigate Internet content with Lisp. The first is to use an http client tool, for example:

https://edicl.github.io/drakma/

https://github.com/fukamachi/dexador

https://github.com/orthecreedence/carrier

https://github.com/fukamachi/fast-http

https://github.com/svenvc/s-http-client

The last four http clients are better than drakma, for their small footprint.

For best results, you should also use a natural language processing system, such as the tagger:

https://github.com/g000001/tagger

Of course, if you use the tagger or another system in Common Lisp, don’t forget to use quicklisp to install. A good example of how to show results and make research reports on the Internet is this COVID-19 data exploration system:

https://github.com/dnaeon/cl-covid19

Of course, what I mentioned above is for amateur systems. If you really want powerful systems, then you’ll need to contact Cycorp:

https://www.cyc.com/

Cycorp offers powerful natural language resources and is being used in large projects, such as the Cleveland Clinic’s natural language interface for biomedical information:

https://my.clevelandclinic.org/