Metadata is omnipresent. You've probably seen it before. You also need to know what it is.
Kubernetes objects are represented by structured data, which is mostly well-organized and sensible. There are notable fields that most of them need, for which "spec" is a good example - primary characteristics determining its behaviour are defined therein. There are immutable fields, for example "status", where the cluster itself maintains a record of the object's current state. There's also one named "metadata", which seems harmless in spite of its importance. There's stuff in there like the object's creation timestamp, unique identifier and similar things that are - like "status" - managed by the cluster and pertain to the very existence of the object itself rather than what that should effect. However, there are also "labels" and "annotations", which play an equally important, albeit different role. Being part of a set of structured data, they're both essentially just key-value pairs, nevertheless it's important to introduce a distinction between "identifying" and "non-identifying" metadata. They're widely employed but not in the same manner - all for good reasons.
Labels - officially named "core grouping primitives" - fall into the former category and are widely used as parameters passed to so-called selectors, determining pods managed by workload controllers, targeted by policies or the nodes eligible to run them. Certain labels are automatically applied to objects, but arbitrary labels may be defined to serve the above purposes. In short, labels are used system-wide to group objects.
Annotations, on the other hand, represent the second category: they're entirely optional but no less useful. I like to refer to them as "post-it stickers" that you can stick on objects and which matter only when read, to those reading them. They double as documentation as well as configuration: the former is normally consulted by API clients or auditors, whereas the latter is generally used to influence the behaviour of components not directly managing the resource in question. (Those managing it will read its "spec" field instead. There are exceptions - cases when certain parameters wouldn't really fit into "spec" - but the above can still be considered a rule of thumb.) Some annotations are applied automatically in line with the above train of thought. In short, annotations are informational notices, also used to configure software.
This concludes our brief foray into metadata territory, paving the way for further advance.
