Webinar
Smarty

Cleaning address databases with Smarty's lead developers

Clean address database
Andrew Townsend
Andrew Townsend
 • 
September 10, 2024
Tags

We recently brought together Kent Gividen and Bryan Amundson, two of Smarty's brilliant lead developers, to discuss the importance of maintaining a clean address database and how Smarty's innovative tools can make this process easier and more efficient.

Question: "How should users prepare their address data before using Smarty's tools?"

Kent: "To begin with, if you have a database that you're looking to clean the addresses for, you can dump those addresses into a CSV file. And there are several ways to make that easier to handle and process."

By dumping your addresses into a CSV file, you create a standardized format that can be easily manipulated, analyzed, and cleaned using Smarty's tools. Kent mentioned that there are several ways to make this process easier, which could include organizing the data in specific ways (such as all data in a single cell or broken into components) and using appropriate delimiters like commas or tabs to separate the data fields.

This preparation step is crucial to ensure that the address cleaning tools can effectively process and validate the data. For example, if you use the Google Sheets or Microsoft Excel plugins, your data must be formatted in a single cell.

Kent and Bryan talked about the value of non-developer tools, but they both suggested SmartyList as a great and underutilized tool.

Question: "Can you explain how the SmartyList command-line tool works?"

Bryan: "SmartyList, or the CLI as some people call it, is a utility used to process CSV files through our APIs. You can get SmartyList on our website.’"

Bryan talked about the data available in Smarty's documentation in regards to the CLI. It's essentially a step by step tutorial on getting running with very little coding.

By using the command-line interface, users can efficiently process large batches of address data without needing a graphical user interface. This method is particularly useful for developers and technical users who are comfortable working in a command-line environment.

Question: "Could you elaborate on the different match modes available and their uses?"

Kent: "We have three different kinds of match columns: strict, invalid, and enhanced."

He went on to give detailed explanations of each match type.

Strict Mode: This mode requires that the input address must closely match a known address in Smarty's database. If there are significant discrepancies or missing components, the address will not be validated. This is useful for ensuring high accuracy and confidence in the validation results.

Invalid Mode: In this mode, even if the address doesn’t match exactly, Smarty will attempt to return as much information as possible about the address. This includes partially valid addresses where some components may be correct, but others might be missing or incorrect. It provides a more lenient approach, offering useful data even when the input is imperfect.

Enhanced Mode: This is a more aggressive and comprehensive matching mode that can handle various errors and missing components in the input address. It includes additional data sources beyond the USPS, covering non-postal addresses and providing more accurate geocoding and address details. Enhanced mode may require an additional license due to the extra data and processing involved.

Kent also discussed how the different match modes can be utilized when accessing Smarty’s APIs. He mentioned that developers can set the match mode parameter in their API requests to control the validation behavior. This is particularly useful for integrating address validation into custom applications and services.

Question: "Why is the CLI so fast, and how does it handle large volumes of data?"

Bryan: "It's multi-threaded which dramatically increases throughput. The CLI will start 10 threads, and batch up to 100 US Street queries in each request."

Multi-threading allows the CLI to handle a high volume of addresses concurrently, significantly reducing the time required to process large datasets. This capability is particularly beneficial for users dealing with millions of addresses.

Reducing the number of API calls decreases the latency and the time spent waiting for responses from the server, further speeding up the process.

Question: "What options do non-programmers have to clean their address databases using Smarty?"

Bryan: "You can use the Bulk Address tool on Smarty's website to process an entire file of addresses. We also offer plugins for Google Sheets and Excel."

These options ensure that non-programmers can efficiently clean and validate their address databases without involving others with technical skills. By leveraging Smarty's user-friendly tools, plugins, and support services, users can maintain accurate and reliable address data, enhancing their overall data quality and operational efficiency.

Subscribe to our blog!
Learn more about RSS feeds here.
rss feed iconSubscribe Now
Read our recent posts
Standardizing country information: Easier said than done
Arrow Icon
At first glance, standardizing country information seems like a straightforward task. After all, how complicated can it be to manage country codes and names? However, once you start diving into the complexities—multiple users, various languages, diacritics, and more—it becomes clear that this process is far from simple. What if Germany has a different name for France than France does because of language barriers between German and French? (They do. The name “France” in German is “Frankreich,” but “France” in French is “France.
National Smarties® Day
Arrow Icon
Well, whaddya know, it’s that time again: National Smarties® Day! You’re probably thinking, “I honestly can’t believe Smarty®, my favorite address verification company, has its own day celebrated on a national scale. ” And if you’re thinking that, let me congratulate you on your finely tuned instincts—the existence of such a day strains belief. We’re talking about Smarties, the candy, people. You know, those tart, powdery discs that resoundingly answer the question, “What if chalk, but delicious?” Care to learn more about National Smarties Day? You’re in luck—there's plenty to unwrap.
The curse of the wrong address
Arrow Icon
Nestled snugly in Orem, Utah, address experts huddle in a circle to hear the cautionary tale of the curse of the wrong address. The lights dim, and a hush falls among the group as the flashlight beam fractures in two, lighting the sides of an older man’s face. He clears his throat and begins…Just days after Melissa Darnitall’s 17th birthday, a Victorian house went up for sale in the quaint town of Rolling Hills. Being historians by trade and visiting the city every year to experience what her mother called “the classic Halloween experience,” Melissa’s parents jumped to buy the property.