René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Bug: An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'","articleBody":"**Describe the bug**\nGot the following error using Semantic Kernel in Java with automatic function calling using 4o* OpenAI models: \"An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'\".\n\n**To Reproduce**\nSteps to reproduce the behavior:\n1. Unzip provided [example](https://github.com/user-attachments/files/17533386/lights-example.zip), build it and execute it:\n```bash\nmvn clean install\nexport OPENAI_API_KEY=sk-...\njava -jar target/lights-example-1.0-SNAPSHOT.jar\n```\n2. Enter two consecute inputs: \"get the lights\" and \"turn them on\"\n3. See error\n\n```\nSLF4J(W): No SLF4J providers were found.\nSLF4J(W): Defaulting to no-operation (NOP) logger implementation\nSLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.\nUser \u003e get the lights\nGetting lights\nAssistant \u003e Here are the current states of the lights:\n\n1. **Table Lamp**: Off\n2. **Porch Light**: Off\n3. **Chandelier**: On\nUser \u003e turn them on\nException in thread \"main\" com.azure.core.exception.HttpResponseException: Status code 400, \"{\n  \"error\": {\n    \"message\": \"An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_Mor7WOBzlb1Ao9ryIczup9mS, call_ejzgT6RAATaWBqmJbrzbzAMP\",\n    \"type\": \"invalid_request_error\",\n    \"param\": \"messages\",\n    \"code\": null\n  }\n}\"\n\tat com.azure.core.implementation.http.rest.RestProxyBase.instantiateUnexpectedException(RestProxyBase.java:388)\n\tat com.azure.core.implementation.http.rest.AsyncRestProxy.lambda$ensureExpectedStatus$1(AsyncRestProxy.java:135)\n\tat reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)\n\tat reactor.core.publisher.MonoUsing$MonoUsingSubscriber.onNext(MonoUsing.java:231)\n\tat reactor.core.publisher.FluxHandle$HandleSubscriber.onNext(FluxHandle.java:126)\n\tat reactor.core.publisher.FluxMap$MapConditionalSubscriber.onNext(FluxMap.java:224)\n\tat reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:113)\n\tat reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:191)\n\tat reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)\n\tat reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839)\n\tat reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:129)\n\tat reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260)\n\tat reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144)\n\tat reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:415)\n\tat reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:439)\n\tat reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:493)\n\tat reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:789)\n\tat reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:114)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat com.azure.core.http.netty.implementation.AzureSdkHandler.channelRead(AzureSdkHandler.java:224)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)\n\tat io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)\n\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)\n\tat io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)\n\tat io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1349)\n\tat io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1389)\n\tat io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)\n\tat io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)\n\tat io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)\n\tat io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)\n\tat io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)\n\tat io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)\n\tat io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)\n\tat io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)\n\tat io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n\tSuppressed: java.lang.Exception: #block terminated with an error\n\t\tat reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:100)\n\t\tat reactor.core.publisher.Mono.block(Mono.java:1742)\n\t\tat org.example.LightsExample.main(LightsExample.java:86)\n```\n\n**Expected behavior**\nThe lights should be turned on.\n\n**Platform**\n - OS: Mac M1\n - IDE: IntelliJ Idea or command line\n - Language: Java\n - Source: Maven repository, version 1.3.0\n\n**Additional context**\nThe problem seems to be reproduced with models 4o and 4o-mini, but not with \"gpt-3.5-turbo-0125\". Change constant `MODEL_ID` in class `LightsExample` to see the difference.[lights-example.zip](https://github.com/user-attachments/files/17546408/lights-example.zip)\n","author":{"url":"https://github.com/miladhub","@type":"Person","name":"miladhub"},"datePublished":"2024-10-28T17:11:35.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/256/semantic-kernel-java/issues/256"}