[大数据]es 异常 org.apache.http.ContentTooLongException: entity content is too long 解决思路

问题：使用es 的java restHighLevelAPI 请求报错如下

Caused by: org.apache.http.ContentTooLongException: entity content is too long [349376897] for the configured buffer limit [104857600]
        at org.elasticsearch.client.HeapBufferedAsyncResponseConsumer.onEntityEnclosed(HeapBufferedAsyncResponseConsumer.java:76)
        at org.apache.http.nio.protocol.AbstractAsyncResponseConsumer.responseReceived(AbstractAsyncResponseConsumer.java:129)
        at org.apache.http.impl.nio.client.MainClientExec.responseReceived(MainClientExec.java:332)
        at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseReceived(DefaultClientExchangeHandlerImpl.java:148)
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.responseReceived(HttpAsyncRequestExecutor.java:224)
        at org.apache.http.impl.nio.client.LoggingAsyncRequestExecutor.responseReceived(LoggingAsyncRequestExecutor.java:112)
        at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:254)
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:73)
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:37)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:113)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:159)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:338)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:316)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:277)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:105)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:584)
        at java.lang.Thread.run(Thread.java:748)

?根据报错可以得到信息：返回的数据大于100MB;因为我的代码设置为

mClient.search(searchRequest, RequestOptions.DEFAULT) // RequestOptions.DEFAULT 为默认值100MB。

既然如此，如何自定义设置这个大小呢？

1.根据返回信息的第一行，最后定位的报错位置 为HeapBufferedAsyncResponseConsumer 类的第76行；双击shift (我的是IDEA编辑器)，将类名输入查找到第76行，定位bufferLimitBytes这个成员变量是谁给它赋值的;

2.通过根据引用为HttpAsyncResponseConsumerFactory第61行new了HeapBufferedAsyncResponseConsumer 进行赋值；

3.HttpAsyncResponseConsumerFactory 是一个接口，寻找它的子类为HeapBufferedResponseConsumerFactory，是通过构造器传入；

4.上面是根据报错的倒推思路；根据我赋值的默认值??RequestOptions.DEFAULT；判断自定义设置肯定也在?RequestOptions类里，看下图小写和第三步的?HttpAsyncResponseConsumerFactory接口是一样的，RequestOptions的构造器如图是私有化的，说明我们是不能直接调用的；下一步追踪是谁调用了这个私有化的构造器

?

5. 发现为该类的内部静态类的build 方法调用的；build方法返回的是一个RequestOptions；而我之前设置的RequestOptions.DEFAULT 也是一个?RequestOptions对象；因此大概可以确定关于参数的设置应围绕 内部静态类 Builder；而Builder 确实包含方法setHttpAsyncResponseConsumerFactory可以自定义参数大小

?

?

?6.? 因此代码为

      val builder = RequestOptions.DEFAULT.toBuilder
      val consumerFactory = new HeapBufferedResponseConsumerFactory(1024 * 1024 * 1024) //1g
      builder.setHttpAsyncResponseConsumerFactory(consumerFactory)
      var searchResponse = mClient.search(searchRequest, builder.build())