Sitelet https://github.com/npgsql/npgsql/issues/3248
Skip to content

Race conditions with Multiplexing=true in 5.0.0-preview1 #3248

Description

@kae

Steps to reproduce

<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup>
		<Authors>kae</Authors>
		<TargetFramework>net5.0</TargetFramework>
		<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
		<OutputType>Exe</OutputType>
	</PropertyGroup>
	<ItemGroup>
		<PackageReference Include="Npgsql" Version="5.0.0-preview1" />
		<PackageReference Include="Dapper" Version="2.0.35" />
	</ItemGroup>
</Project>
using System;
using System.Linq;
using System.Threading.Tasks;
using Dapper;

namespace pgTest {
	public class Program {
		private static readonly string connStr =
			"Server=db-host;Port=5432;Database=test_db;User id=test;Password=test;Multiplexing=true";

		public static async Task Test(int i) {
			try {
				using var conn = new Npgsql.NpgsqlConnection(connStr);
				await conn.OpenAsync();
				using var tx = conn.BeginTransaction(System.Data.IsolationLevel.ReadCommitted);
				var rs = await conn.QueryFirstOrDefaultAsync<int>("select 1", transaction: tx);
				tx.Commit();
			}
			catch (Exception e) {
				Console.WriteLine(e);
			}
		}

		public static async Task Main() {
			try {
				await Task.WhenAll(Enumerable.Range(0, 1000).Select(i => Test(i)));
			}
			catch (Exception e) {
				Console.WriteLine(e);
			}
		}
	}
}

The issue

for Enumerable.Range(0, 100) it's throwing in 1 out of 5-6 runs

Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing'
   at Npgsql.NpgsqlConnector.<StartUserAction>g__DoStartUserAction|205_0(<>c__DisplayClass205_0& )
   at Npgsql.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command)
   at Npgsql.NpgsqlConnection.BeginTransaction(IsolationLevel level, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlConnection.BeginTransaction(IsolationLevel level)
   at Npgsql.NpgsqlConnection.BeginTransactionAsync(IsolationLevel level, CancellationToken cancellationToken)
--- End of stack trace from previous location ---
   at pgTest.Program.Test(Int32 i) in C:\Users\kae\Dump\dnx\pgTest\Program.cs:line 15

And for Enumerable.Range(0, 1000) it hangs for a minutes and than gets multiple pool exhaustion exceptions like

Npgsql.NpgsqlException (0x80004005): The connection pool has been exhausted, either raise MaxPoolSize (currently 100) or Timeout (currently 15 seconds)
   at Npgsql.ConnectorPool.<>c__DisplayClass38_0.<<Rent>g__RentAsync|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.NpgsqlConnection.<>c__DisplayClass148_0.<<StartBindingScope>g__StartBindingScopeAsync|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.ConnectorPool.BootstrapMultiplexing(NpgsqlConnection conn, NpgsqlTimeout timeout, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlConnection.<>c__DisplayClass41_0.<<Open>g__BootstrapMultiplexing|1>d.MoveNext()
--- End of stack trace from previous location ---
   at pgTest.Program.Test(Int32 i) in C:\Users\kae\Dump\dnx\pgTest\Program.cs:line 14

followed with

Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing'
   at Npgsql.NpgsqlConnector.<StartUserAction>g__DoStartUserAction|205_0(<>c__DisplayClass205_0& )
   at Npgsql.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command)
   at Npgsql.NpgsqlConnector.Rollback(Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlTransaction.Rollback()
   at Npgsql.NpgsqlTransaction.Dispose(Boolean disposing)
   at System.Data.Common.DbTransaction.Dispose()
   at pgTest.Program.Test(Int32 i) in C:\Users\kae\Dump\dnx\pgTest\Program.cs:line 17

and

Npgsql.NpgsqlException (0x80004005): Exception while reading from stream
 ---> System.IO.IOException: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine..
 ---> System.Net.Sockets.SocketException (10053): An established connection was aborted by the software in your host machine.
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at Npgsql.NpgsqlReadBuffer.<>c__DisplayClass37_0.<<Ensure>g__EnsureLong|0>d.MoveNext()
   at Npgsql.NpgsqlReadBuffer.<>c__DisplayClass37_0.<<Ensure>g__EnsureLong|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.NpgsqlConnector.<>c__DisplayClass170_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.NpgsqlConnector.<>c__DisplayClass170_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.Consume(Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.Close(Boolean connectionClosing, Boolean async)
   at Npgsql.NpgsqlConnector.CloseOngoingOperations(Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlTransaction.Dispose(Boolean disposing)
   at System.Data.Common.DbTransaction.Dispose()
   at pgTest.Program.Test(Int32 i) in C:\Users\kae\Dump\dnx\pgTest\Program.cs:line 17

and

System.NullReferenceException: Object reference not set to an instance of an object.
   at Npgsql.NpgsqlConnector.<>c__DisplayClass170_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.NpgsqlConnector.<>c__DisplayClass170_0.<<DoReadMessage>g__ReadMessageLong|0>d.MoveNext()
--- End of stack trace from previous location ---
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
   at Dapper.SqlMapper.QueryRowAsync[T](IDbConnection cnn, Row row, Type effectiveType, CommandDefinition command) in /_/Dapper/SqlMapper.Async.cs:line 483
   at pgTest.Program.Test(Int32 i) in C:\Users\kae\Dump\dnx\pgTest\Program.cs:line 16

Activity

  1. vonzshik commented on Oct 21, 2020

    @vonzshik
    Contributor

    The first error is most likely due to #3249.

    While I do have a pr ready for the other errors, I still would like to take a closer look at the exact reason.

  2. vonzshik commented on Oct 22, 2020

    @vonzshik
    Contributor

    @roji so, I've managed to locate the issue. The problem is with the idleChanel in the ConnectorPool.
    For some reason, if there are too many parallel threads, its performance is degrading to the floor. So there comes a time, when we have 100 idle connectors, and there is nothing in the channel reader. And it takes around 0.5 seconds to read something from the channel.

  3. added this to the 5.0.0 milestone on Oct 25, 2020
  4. added theissue type on Jun 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions