Other answers reference the documentation for the Item property. It might not be immediately obvious why they are relevant looking at the following code snippet.
ConfigurationManager.AppSettings["blah"]
The square bracket syntax is used in C# to access indexers. These are special properties that allow a class to be indexed in the same way that an array can be. Looking at the definition of the NameValueCollection.Item property, you will notice that it does not use the normal property syntax. The this keyword and the indexer parameters are used to define this property as an indexer.
public string this[
string name
] { get; set; }
In the documentation, indexers are implicitly named Item and parameters are surrounded by square brackets.
It is not clear to me why there were answers that referenced the Get method - maybe one calls the other?
At any rate, to answer the question...
No. An exception will not be thrown if you access a non-existent key - a null will be returned.
Here is the relevant section from the NameValueCollection.Item property documentation.
This property returns null in the following cases: 1) if the specified key is not found; and 2) if the specified key is found and its associated value is null. This property does not distinguish between the two cases.