You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
EnvironmentVariablesConfigurationProvider should support all types of connectionstrings Β #36123
When setting a connection string on an Azure Webapp using Azure CLI the type must be specified.
The following types are supported: ApiHub, Custom, DocDb, EventHub, MySql, NotificationHub, PostgreSQL, RedisCache, SQLAzure, SQLServer, ServiceBus az webapp config connection-string set
This means that if I use Configuration.GetConnectionString("MyServiceBusConnectionString") and I have added a connection string of type ServiceBus on my Azure Website with the same name the code will unexpectedly fail. I will have to define the type of the connectionstring in azure to be Custom for this to work.
It would be much better if EnvironmentVariablesConfigurationProvider and the method AzureEnvToAppEnv supported the same types as the Azure CLI does.
These are the environment variable prefixes that I think should be added:
APIHUBCONNSTR_
DOCDBCONNSTR_
EVENTHUBCONNSTR_
NOTIFICATIONHUBCONNSTR_
POSTGRESQLCONNSTR_
REDISCACHECONNSTR_
SERVICEBUSCONNSTR_
I would be happy to make a PR for this should you agree that this is something that should be supported.
Reacted by Leonardo Pignataro, pinkfloydx33, Marcus OtterstrΓΆm, Facundo Pugliese, Alvin Dutt, Eirik F. Wahl, Tushar Raj, Simon Wimmesberger, HamzaAboumousa, spaasis and 1 more
Wow, that layering is really bizarre. I'd have expected there to be a library that read from config and dealt with this, not having it embedded in the provider itself.
Do we actually use those provider names for anything? I did a quick search across Extensions, AspNetCore, and EF Core, and outside of configuration tests, the _ProviderName config doesnβt appear to be used. What are those for?
And why would this configuration provider decide about what database provider I use? For example, itβs just by chance that MySqlConnector uses the same namespace for their MySQL provider (for migration and compatibility reasons). But they didnβt have to do it like that.
Adding a note because we got a report via email that a customer hit the fact that we don't read the connection string correctly for PostgreSQL when it is configured in app services. If this issue was fixed, the simple code they used would work as they expect and they would get the connection string from the value configured in Azure App Services:
Possibly the simplest workaround is to use the "custom" option instead.
To answer to previous comments:
@poke No framework code that I am aware currently uses _ProviderName , but applications can do it if the wish to. For example an application could support using different kinds of databases with the same code and resolve the DbProviderFactory from DbProviderFactories using the invariant name.
@Tratcher I am not sure I like the layering either but the logic needs to go somewhere. My main complaint is that the logic is bound to become stale anytime Azure App Services adds new connection string types. Ideally they would publish a package that encapsulates the logic. But this way we own it and it is easier for us to make it right.
@bradygaster, I was told by @glennc that you will be following up on this. I will add you to the email thread about the customer issue.
Reacted by Shay Rojansky, Emil Tang Kristensen, Leonardo Pignataro, Nicolai SchΓΆnberg, Oziel GuimarΓ£es de Paula Silva and Facundo Pugliese
@divega can we at least look at properly re-layering this in 5.x? E.g. move the connection strings out of the environment variable provider to their own API that consumes config and is called from the templates. At the very least it would be easier to maintain and ship an out of band leaf node dependency.
@Tratcher I think you can make a proposal, check with @bradygaster, @DamianEdwards that the resulting experience (and possible breaking change) is worth it.
Eeeek, having strings that are hard-coded to an Azure service's set of supported things in this provider is making me feel poorly. We shouldn't definitely look at this in 5.0
absolutely agree with @DamianEdwards on this - I don't see those prefixes adding value, and have been confused by their existence for some time. I'd like to re-evaluate this, and hopefully find a way to stop doing it. IMO, a "connection string is a connection string," and shouldn't contain any "triggering behavior" for the consumer or configurator. Screen shot from the AZ CLI below for reference. Adding @btardif here from App Service team as he's on the email thread in which we're discussing this internally. @btardif is this a requirement of the web apps side or something imposed by the CLI side?
Al - is there any reason why we can't agree "a connection string is a connection string" and figure out if unwinding the prefixes is possible?
I believe the original reason for the prefix was because those got projected into the .NET Framework configuration system as provider specific connection strings.
There's no first class concept like that in .NET Core so we mapped it internally (in a hacky way) to ConnectionString:{Name} or ConnectionString:{Name}_{Provider}. So the options here are to:
Change AppService to match our conventions or add our conventions to the list natively when connection strings appear.
Create an app service specific configuration provider and have users add that (or light it up on app service)
Just to mention the point discussed offline, having a provider classification on a connection string could allow us to do interesting things in the UI, e.g. a provider-specific UI for building the string, validation, or whatever. There's definitely value in that, especially for people starting out and not familiar with what options are supported in the connection string of which provider, etc.
I created a stock "Web App with Postgres" in Azure and deployed my application, only to spend hours trying to debug why the app wasn't picking up my connection string (which had been automatically added to the web app using the "PostgreSQL" type).
That lead me to eventually find this issue and switch the connection string type to "Custom" instead, and suddenly the app worked fine. What a mess π
I would like to see this issue addressed if possible, or perhaps an explicit warning for those of us who select an unsupported connection string type such as PostgreSQL if that'd be easier.
Reacted by Andrew Bullock, Emil Tang Kristensen, Leonardo Pignataro and Facundo Pugliese
Fact: The auto-generated (semantically correct) configuration by the Azure Web App wizard that creates a Postgres database and web app in one step is just not working.
Why the template for that wizard is not simply forcing a type of "Custom" for the connection string and adds a short note somewhere on this known issue, is beyond me. Either nobody of the responsibles has ever tested this with even just a "hello world"-type of application, or they have and simply don't care. Microsoft rather lets everyone figure it out by themselves (since 2019!). Losing hours to a problem like this is frustrating.
var connectionString = builder.Configuration.GetConnectionString("POSTGRESQLCONNSTR_x");
nit: with GetConnectionString() API, it should be just "x"; the full form"POSTGRESQLCONNSTR_x" can be used with indexer ([]).
Meanwhile, we can inherit from EnvironmentVariablesConfigurationProvider and add the missing prefixes.
Source
usingSystem;usingSystem.Collections;usingMicrosoft.Extensions.Configuration;usingMicrosoft.Extensions.Configuration.EnvironmentVariables;internalstaticclassCustomEnvironmentVariablesExtensions{publicstaticIConfigurationBuilderAddCustomEnvironmentVariables(thisIConfigurationBuilderbuilder,string?prefix=null){returnbuilder.Add(newCustomEnvironmentVariablesConfigurationProvider.Source(prefix));}}fileclassCustomEnvironmentVariablesConfigurationProvider(string?prefix):EnvironmentVariablesConfigurationProvider(prefix){privateconststringPostgreSqlPrefix="POSTGRESQLCONNSTR_";privateconststringNotificationHubPrefix="NOTIFICATIONHUBCONNSTR_";privateconststringServiceBusPrefix="SERVICEBUSCONNSTR_";privateconststringEventHubPrefix="EVENTHUBCONNSTR_";privateconststringDocumentDbPrefix="DOCDBCONNSTR_";privateconststringRedisCachePrefix="REDISCACHECONNSTR_";publicoverridevoidLoad(){base.Load();LoadAdditionalPrefixes(Environment.GetEnvironmentVariables());}privatevoidLoadAdditionalPrefixes(IDictionaryenvVariables){foreach(DictionaryEntryentryinenvVariables){stringkey=(string)entry.Key;string?value=(string?)entry.Value;if(key.StartsWith(PostgreSqlPrefix,StringComparison.OrdinalIgnoreCase)){HandleConnectionStringPrefix(PostgreSqlPrefix,key,value,"Npgsql");}elseif(key.StartsWith(NotificationHubPrefix,StringComparison.OrdinalIgnoreCase)){HandleConnectionStringPrefix(NotificationHubPrefix,key,value);}elseif(key.StartsWith(ServiceBusPrefix,StringComparison.OrdinalIgnoreCase)){HandleConnectionStringPrefix(ServiceBusPrefix,key,value);}elseif(key.StartsWith(EventHubPrefix,StringComparison.OrdinalIgnoreCase)){HandleConnectionStringPrefix(EventHubPrefix,key,value);}elseif(key.StartsWith(DocumentDbPrefix,StringComparison.OrdinalIgnoreCase)){HandleConnectionStringPrefix(DocumentDbPrefix,key,value);}elseif(key.StartsWith(RedisCachePrefix,StringComparison.OrdinalIgnoreCase)){HandleConnectionStringPrefix(RedisCachePrefix,key,value);}}}privatevoidHandleConnectionStringPrefix(stringprefix,stringfullKey,string?value,string?provider=null){stringnormalizedKey=Normalize(fullKey.Substring(prefix.Length));Data[$"ConnectionStrings:{normalizedKey}"]=value;if(provideris not null){Data[$"ConnectionStrings:{normalizedKey}_ProviderName"]=provider;}}privatestaticstringNormalize(stringkey)=>key.Replace("__",ConfigurationPath.KeyDelimiter);internalclassSource(string?prefix):IConfigurationSource{publicIConfigurationProviderBuild(IConfigurationBuilderbuilder){returnnewCustomEnvironmentVariablesConfigurationProvider(prefix);}}}
usage:
usingSystem;usingMicrosoft.Extensions.Configuration;IConfigurationRootconfiguration=newConfigurationBuilder().AddCustomEnvironmentVariables().Build();Console.WriteLine($"PostgreSQL Connection String with GetConnectionString: {configuration.GetConnectionString("x")}");Console.WriteLine($"PostgreSQL Connection String with indexer: {configuration["POSTGRESQLCONNSTR_x"]}");
run with dotnet 9.0:
$ export POSTGRESQLCONNSTR_x="Server=myServer;Database=myDb;User=myUser;Password=myPass;"
$ dotnet add package Microsoft.Extensions.Configuration.EnvironmentVariables
$ dotnet run
PostgreSQL Connection String with GetConnectionString: Server=myServer;Database=myDb;User=myUser;Password=myPass;
PostgreSQL Connection String with indexer: Server=myServer;Database=myDb;User=myUser;Password=myPass;
Wow, just spent half a day trying to pinpoint this issue for the "Redis Cache" connection string type in Azure App Services.
Pretty frustrating that now for more than 5 years so many people must be running into this!
I love being subscribed to this issue still 6 years later, knowing how much time i wasted on this, seeing how noone cares, and being glad i migrated away from this diabolical stack to other platforms with their own cryptic and less documented quirks that make me hate them equally
So long as we change in sync, I guess it's OK. Who adds the upstream usage of these? Can we drop a breadcrumb there to remind them to update this library whenever something new is added?
Activity
Tratcher commentedon May 9, 2019
Wow, that layering is really bizarre. I'd have expected there to be a library that read from config and dealt with this, not having it embedded in the provider itself.
poke commentedon May 28, 2019
Do we actually use those provider names for anything? I did a quick search across Extensions, AspNetCore, and EF Core, and outside of configuration tests, the
_ProviderNameconfig doesnβt appear to be used. What are those for?And why would this configuration provider decide about what database provider I use? For example, itβs just by chance that MySqlConnector uses the same namespace for their MySQL provider (for migration and compatibility reasons). But they didnβt have to do it like that.
divega commentedon Sep 25, 2019
Adding a note because we got a report via email that a customer hit the fact that we don't read the connection string correctly for PostgreSQL when it is configured in app services. If this issue was fixed, the simple code they used would work as they expect and they would get the connection string from the value configured in Azure App Services:
Possibly the simplest workaround is to use the "custom" option instead.
To answer to previous comments:
@poke No framework code that I am aware currently uses
_ProviderName, but applications can do it if the wish to. For example an application could support using different kinds of databases with the same code and resolve the DbProviderFactory from DbProviderFactories using the invariant name.@Tratcher I am not sure I like the layering either but the logic needs to go somewhere. My main complaint is that the logic is bound to become stale anytime Azure App Services adds new connection string types. Ideally they would publish a package that encapsulates the logic. But this way we own it and it is easier for us to make it right.
@bradygaster, I was told by @glennc that you will be following up on this. I will add you to the email thread about the customer issue.
Tratcher commentedon Sep 25, 2019
@divega can we at least look at properly re-layering this in 5.x? E.g. move the connection strings out of the environment variable provider to their own API that consumes config and is called from the templates. At the very least it would be easier to maintain and ship an out of band leaf node dependency.
divega commentedon Sep 25, 2019
@Tratcher I think you can make a proposal, check with @bradygaster, @DamianEdwards that the resulting experience (and possible breaking change) is worth it.
DamianEdwards commentedon Oct 1, 2019
Eeeek, having strings that are hard-coded to an Azure service's set of supported things in this provider is making me feel poorly. We shouldn't definitely look at this in 5.0
@davidfowl @glennc
bradygaster commentedon Oct 3, 2019
absolutely agree with @DamianEdwards on this - I don't see those prefixes adding value, and have been confused by their existence for some time. I'd like to re-evaluate this, and hopefully find a way to stop doing it. IMO, a "connection string is a connection string," and shouldn't contain any "triggering behavior" for the consumer or configurator. Screen shot from the AZ CLI below for reference. Adding @btardif here from App Service team as he's on the email thread in which we're discussing this internally. @btardif is this a requirement of the web apps side or something imposed by the CLI side?
Al - is there any reason why we can't agree "a connection string is a connection string" and figure out if unwinding the prefixes is possible?
davidfowl commentedon Oct 3, 2019
I believe the original reason for the prefix was because those got projected into the .NET Framework configuration system as provider specific connection strings.
There's no first class concept like that in .NET Core so we mapped it internally (in a hacky way) to
ConnectionString:{Name}orConnectionString:{Name}_{Provider}. So the options here are to:roji commentedon Oct 3, 2019
Just to mention the point discussed offline, having a provider classification on a connection string could allow us to do interesting things in the UI, e.g. a provider-specific UI for building the string, validation, or whatever. There's definitely value in that, especially for people starting out and not familiar with what options are supported in the connection string of which provider, etc.
khellang commentedon Mar 12, 2020
Wow, this was a bizarre experience.
I created a stock "Web App with Postgres" in Azure and deployed my application, only to spend hours trying to debug why the app wasn't picking up my connection string (which had been automatically added to the web app using the "PostgreSQL" type).
That lead me to eventually find this issue and switch the connection string type to "Custom" instead, and suddenly the app worked fine. What a mess π
tnc1997 commentedon May 8, 2020
I would like to see this issue addressed if possible, or perhaps an explicit warning for those of us who select an unsupported connection string type such as PostgreSQL if that'd be easier.
34 remaining items
pk-ot commentedon Jan 28, 2024
Fact: The auto-generated (semantically correct) configuration by the Azure Web App wizard that creates a Postgres database and web app in one step is just not working.
Why the template for that wizard is not simply forcing a type of "Custom" for the connection string and adds a short note somewhere on this known issue, is beyond me. Either nobody of the responsibles has ever tested this with even just a "hello world"-type of application, or they have and simply don't care. Microsoft rather lets everyone figure it out by themselves (since 2019!). Losing hours to a problem like this is frustrating.
But hey, now it's scheduled for .NET 9. π
sqbiak commentedon Nov 23, 2024
Hello from a 2024.... from a S&P 500 where I wanted to fire a guy for this code:
but then a simple
didn't work.
am11 commentedon Nov 23, 2024
nit: with
GetConnectionString()API, it should be just"x"; the full form"POSTGRESQLCONNSTR_x"can be used with indexer ([]).Meanwhile, we can inherit from
EnvironmentVariablesConfigurationProviderand add the missing prefixes.Source
usage:
run with dotnet 9.0:
cliedeman commentedon Nov 29, 2024
Just ran into this using Cosmos db with the
Document DbtypeSebastianToet commentedon Feb 18, 2025
Wow, just spent half a day trying to pinpoint this issue for the "Redis Cache" connection string type in Azure App Services.
Pretty frustrating that now for more than 5 years so many people must be running into this!
spaasis commentedon Apr 24, 2025
Six years later...
txnnr commentedon Apr 25, 2025
ran into this issue using PostgreSQL as well
trullock commentedon Apr 25, 2025
I love being subscribed to this issue still 6 years later, knowing how much time i wasted on this, seeing how noone cares, and being glad i migrated away from this diabolical stack to other platforms with their own cryptic and less documented quirks that make me hate them equally
Estyn commentedon Apr 25, 2025
We also just ran into this issue.
davidfowl commentedon May 2, 2025
OK lets just add them, we can hopefully remove them in the long run. This is a VERY unsavory hack, but maybe we just embrace it at this point.
ericstj commentedon May 3, 2025
So long as we change in sync, I guess it's OK. Who adds the upstream usage of these? Can we drop a breadcrumb there to remind them to update this library whenever something new is added?
tarekgh commentedon May 27, 2025
I have opened the PR #116037 if anyone interested to review.