With the recent advancements in LLMs, discussions about automating the annotation process have become increasingly common. It's easy to fall into the trap of assuming that ML models can simply replace the annotation and labeling work traditionally done by humans. This paper explains the context, quality, and desired outcomes you should consider for your business when the topic arises.