<style> h3 { border-bottom: 1px solid #ccc; } section { margin-bottom: 2rem; padding: 0em 1em; padding-bottom: 1em; border-radius: 4px; background-color: #f7f7f7; border: 1px solid #ccc; } summary { font-weight: bolder; } summary:hover { text-decoration: underline; } .todo { color: #ff00ff; border: 2px dashed #ff00ff; padding: 0em 1em; border-radius: 5px; //display: none; // UNCOMMENT TO HIDE TODOs } </style> # Lab 9: LLMs Part 2 (Fall 2025) This week, we continue building the mystery flight recommendation system and learning another common programming tool: regular expressions. We'll learn how to have more sophisticated interactions with LLMs from within programs. :::warning This is our last lab of CS200!! ::: # Learning Objectives - Experience building a program off someone else's codebase - Learn and practice using regular expressions - Provide conversation history to an LLM when calling it as a service # Setup <!-- :::danger Elijah/TAs -- please fill in anything they need to know about getting their swap code from Peerceptiv ::: --> This week, you'll begin with another group's code from Lab 8. Continue using your own environment and the tests you created last week. If you didn’t complete the [Lab 8](https://docs.cs200.io/We3vTrSPRlWef1DwaNwHwQ) setup, do that first. :::info Stay with the same lab partners from last week! If you cannot, your TA will find another unmatched partner. ::: ### Getting a Peer Submission 1. The group member who submitted last week's code (or one member if everyone submitted) should go to the [Peerceptiv assignment](https://app.peerceptiv.com/course/685d4fd5-50f2-41a6-a8f5-24507d336f20/assignment/65a7bb50-e6ae-4d78-87c7-1c1028522d1c/dashboard) and start a review. Peerceptiv should provide another group’s submission—download it and share it with the group. If successful, skip to [Setting up the New Code](#Setting-up-the-New-Code). 2. If step 1 fails, another group member should try starting a review. Peerceptiv should allow them to access a submission even if they didn’t submit previously. Download and share it. Repeat this process until a submission is retrieved or all members have tried. If successful, continue to [Setting up the New Code](#Setting-up-the-New-Code). 3. If no valid submission is available via Peerceptiv, use the [backup submission](https://drive.google.com/file/d/1wkZ8nqqMwx9S6UdaBin2d3zK6wMGI9Qt/view?usp=drive_link) provided. In this case, submit your review via the designated Google Form instead of Peerceptiv. **Don't answer the review questions yet!** You'll complete those at the [end of the lab](#Task-12). ### Setting up the New Code In your Lab 8 directory: 1. Rename your existing `flight_chat.py` --> `my_lab8_flight_chat.py` 2. Create two new empty files: `flight_chat.py` and`lab8_flight_chat.py` 3. Copy the contents of the downloaded submission code into both new files `flight_chat.py`&`lab8_flight_chat.py` **Note:** If the code was copied from a PDF, extra newlines may have been added. Double-check for syntax errors before proceeding. # Review the code that you received <section> ### Task 1 Comment initial notes at the bottom of the code you received. Does it seem broken out well into helpers, well commented, with clear names on functions and variables? How are their LLM prompts? Are there parts of the code that do not work, especially with edge cases? (you'll provide an actual review at the end of the lab). </section> <section> ### Task 2 Get the code you received working through city selection from the first part (you can skip the cleanup for now) before you move on. If you inherited code that hasn't finished this part, but your code did, feel free to copy the solution over. :::info Default to using the non-JSON version of the get_date() helper method for this lab. ::: :::warning For the [Tunnelblick](https://cs.brown.edu/about/system/connecting/openvpn/) connection, use `fmlast_browncs`. If you can see http://perfmon.cs.brown.edu/, you are all set! Still have one connection per group in a big lab, and work off of that person's laptop! ::: </section> # Selecting flights Now we want to augment the program to return a candidate flight to the user. <section> ### Task 3 Search the provided `flights200.csv` file (from last week's stencil) for a flight that matches the date and city provided by a user. Select any matching flight at random. You can just print an informative message if no flight exists. For now, print just the flight code (like `"ZA453"`) from the csv file. </section> We're intentionally not telling you how to do this. Use a combination of your programming knowledge and a code-generation tool to get this working. Consider developing a plan for this, then deciding which plan elements you could reasonably get the code-generator to do for you. If you aren't sure how to proceed, the TAs will ask to see your plan-in-progress. :::success **Checkpoint:** Call over a TA to check your work ::: ## Describing available flights In the [original sample interaction](https://docs.cs200.io/I74H-Y0uTgOoxl0eQf8sbw?view#First-the-big-picture), the following message summarized a flight to the user as follows: ``` How about Zinnia Air flight 726 to New York? ``` The flight that you retrieved from the csv file didn't show the name of the airline. Instead, it showed a two-letter airline code followed by the flight number. To generate this message, we need to extract the airline code and look up the real airline name from the flight data. Your first task is to extract the airline code. # Regex :::info This component does not require the VPN - work together but don't feel the need to work on the same computer! ::: Regular expressions are powerful tools for pattern matching and extraction. In this section, you’ll explore how to leverage regex to dissect structured strings into their meaningful components. As you work through the exercises, you’ll see how a simple regex can evolve into a critical component of larger tasks—like parsing flight information from a string. <section> ### Task 4 1. Go to [regex101.com](https://regex101.com), which is an awesome site for building and testing regular expressions. 2. Put the following string into the "Test String" box ``` The cat in the hat was late, and that at the CSCI0200 lecture with bat9. ``` 3. Enter the pattern string `cat` into the "Regular expression" box, like this: ____ 4. At this point, your regex101 window should look like this (see annotations or details)  5. Notice that the _cat_ string is highlighted in the text--**this shows what part of the text matched your regex**! If you don't see `cat`highlighted, make sure your regex101 window looks like the figure above (make sure "Python" and "Match" are checked at the left). If you still have issues, call over a TA for help. </section> Now that you’ve seen how a fixed pattern like cat is matched, let’s advance to using character sets to allow for variation within the pattern. <section> ### Task 5 Now let’s start building up patterns where not all characters are fixed. Let’s say we want to look for every three-letter word containing “at” (e.g., “cat,” “bat,” “sat,” etc.). We can use the pattern: ```[a-z]at``` Here, the square brackets `[a-z]` define a set/range of characters (all lowercase letters), immediately followed by `at`. So `[a-z]at` will match exactly three-letter strings ending in “at.” Put this new regex into the site and see what it matches. </section> <section> ### Task 6 Now suppose we want to find every word ending in “at,” regardless of length (e.g., “at,” “bat,” “seat,” etc.). We can introduce a quantifier—the most common of which is `*`. ```[a-z]*at``` The `*` quantifier says “zero or more occurrences of the preceding element.” Here, that means “zero or more lowercase letters” followed by at. As a result, this regex will match “at” by itself (because zero letters are allowed before “at”) as well as larger words like “seat” or “great.” Put this new regex into the site and see what it matches. </section> In Task 6, we introduced the `*` quantifier for flexible matching. However, this sometimes matches too much. In Task 7, we switch to the `+` quantifier to ensure there’s at least one character before 'at', providing more control over our matches. <section> ### Task 7 If we want to exclude “at” so that at least one character appears before “at,” we can use `+` instead of `*`: ```[a-z]+at``` The `+` quantifier requires “one or more” occurrences of the preceding element. Thus, `[a-z]+at` will not match “at” by itself (because zero letters are not allowed) but will still match words like “cat,” “that,” or “great.” Even so, it might match more than you intend—for instance, it would match the “lat” in “late.” So we sometimes need a way to ensure “at” appears at the end of a word. </section> This regex still has an issue: it matches “lat” in “late.” To ensure “at” truly appears at the end of a word, add a word boundary `(\b)`. A word boundary forces “at” to end before any non-word character (like punctuation, spaces, or digits), so only full words ending in “at” will match. <section> ### Task 8 To ensure our match ends exactly at a word boundary, we use `\b`. This prevents characters like digits, underscores, or letters from continuing after “at”: ```[a-z]*at\b``` - `[a-z]*`: zero or more lowercase letters - `at`: the literal string “at” - `\b`: a word boundary (no letters/digits/underscores after) </section> If we want to avoid matches like “that” or “great”, we could add a word boundary to the beginning as well, for a final regex of `\b[a-z]at\b`. # Describing available flights cont. Having explored various regex techniques for matching different word patterns, we now apply these techniques to a practical problem. Task 9 will use regex to extract components from a flight code and construct a user-friendly flight summary. <section> ### Task 9 Once a random flight has been selected, use regular expressions to separate the airline code from the flight number. Use the pieces to construct the flight summary message to the user with the following format: How about Zinnia Air flight 726 to New York? The starter file `airline-codes.csv` shows the full name of the airline for each two letter code. Some useful hints about some symbols: - `^` – Marks the start of the string (no partial matches from the middle). - `[A-Z]{2}` – Matches exactly two uppercase letters (likely your airline code). - `(\d+)` – Captures one or more digits (the flight number). - `$` – Marks the end of the string (no extra characters allowed after the digits). - Hint: You might want to look into [re.MatchObject.group()](https://www.geeksforgeeks.org/python/re-matchobject-group-function-in-python-regex/)! :::warning You might have to edit your code from [Task 3](#Task-3) to get the destination city! ::: </section> :::success **Checkpoint:** Call over a TA to check your work ::: # Finer-grained control of LLM prompts While doing part 1, some of you may found that you need a lot of subtleties in the date-conversion prompt. In practice, we refine answers by having conversations with LLMs, not just by sending single messages. When you call out to an LLM through a url, however (rather than in a back and forth conversation), all that prior conversation gets lost. Accordingly, the parameters when calling an LLM via a function allow you to provide some prompt/response history on which the LLM can base an answer. Look in the `llm_service.py` file. You'll see the following function ```python def get_date_conversion_messages(msg): return [ { "role": "user", "content": msg, }, ] ``` Notice how this function is returning a list? This list represents the message history we want to pass to the LLM. Here's an example with more history: ```python def get_date_conversion_messages(msg): return [ { "role": "system", "content": msg, }, { "role": "user", "content": "jan17", }, { "role": "assistant", "content": "2025-01-17", }, { "role": "user", "content": "5/4/23", }, { "role": "assistant", "content": "2023-05-04", }, ] ``` Here, "user" marks message from the user and "assistant" marks responses from the LLM. This example is seeding the history of the LLM with previous prompts and good answers. <section> ### Task 10 Edit the `llm_service.py` file to make a copy of the `get_date_conversion_message`. In the copy, try adding some chat history in the spirit of what's shown in the handout. Experiment and see if you can find some prompts that fail to convert properly with the original version, but do pass with the one that provides more context. Our entire goal with this is to have to explore and learn how llm programming works. There's no specific goal here other than you get some experience with using these tools. </section> <section> ### Task 11 Do any cleanup that you feel is necessary on your code, especially if what you inherited wasn't particularly well structured. Again, we're asking you to do this for the experience of working with inherited code, which you'll end up doing if you go on in CS. [Here](https://drive.google.com/file/d/1y01YAZehPyCRR98FjfEN2vw0m4_KjuHU/view) is an example of a well structured partial solution to this flight system for you to refer to. </section> <section> ### Task 12 Go back and add to your answer from the [start this lab](#Task-1), further reviewing the `lab8_flight_chat.py` code you originally received. Write together and make one submission per group. 1. How did the code you received compare to your own? - What do you wish you had done differently in your code to make it easier to understand or modify? - What do you wish the authors of the code you received had done differently (if anything)? 2. Compare and contrast the embedded LLM prompts in the code you received with your own. Did their prompts give you any ideas for making yours more effective or easier to maintain? 3. What bugs or unexpected behavior did you encounter—either in the code you received from another group or while working through the rest of this lab? How might you write test cases to catch that kind of issue? Can you give an example? Only the partner that received a code submission through Peerceptiv should complete the review on [Peerceptiv](https://app.peerceptiv.com/course/685d4fd5-50f2-41a6-a8f5-24507d336f20/assignment/65a7bb50-e6ae-4d78-87c7-1c1028522d1c/dashboard). If you used the backup code submission in Google Drive, please complete the review using [this Google form](https://docs.google.com/forms/d/e/1FAIpQLSfFyZuKggKmHM2u-YMCUcbu1_Hyna06-azSA85AwBRtUdMh0w/viewform?usp=publish-editor). </section> <section> ### Task 13 Finally, please upload your final code to [Gradescope](https://www.gradescope.com/courses/1101102/assignments/6593115). Don't worry, we aren't grading it. This is a relatively new lab, and Kathi wants to be able to see what students produced (to see what modifications we might want to make for future semesters). Only one person needs to upload without adding any names. </section> :::success **Checkpoint:** Congratulations, you’ve completed this week’s lab! (All labs in fact!) Check in with a TA, and then you can go home. Nice job! ::: ## **💡Applications💡** - **Natural Language Parsing:** LLM-based prompts let you convert free-form user input (dates, locations, preferences) into structured formats without hand-coding every rule. - **Pattern Extraction:** Regular expressions power log parsing, data validation, and extracting structured fields (like flight codes) from arbitrary strings. - **Contextual AI Prompts:** Supplying conversation history to an LLM improves reliability by teaching the model your desired format through examples. - **Fuzzy Matching:** String-difference libraries (e.g., `difflib`) enable “did you mean?” features for typos, autocompletion, and search suggestions. - **JSON Data Exchange:** JSON remains the lingua franca for passing structured data between services, scripts, and front-end/back-end components. *** _Please let us know if you find any mistakes, inconsistencies, or confusing language in this or any other CSCI0200 document by filling out the_ [_anonymous feedback form_](https://docs.google.com/forms/d/e/1FAIpQLSdmeYWU8_8v9fizxUs_3Fq6Ddsn8Tmas3tkL63ALA6XaZ5Tuw/viewform)_._