Hi Hellojoy,
Can you please try 1.1.5?
https://github.com/xucito/logstash-output-spoof/releases/tag/0.1.5
The issue should be resolved now.
Regards,
Tony Nguyen
java.nio.BufferUnderflowException
at org.jnetpcap.util.checksum.Checksum.crc32IEEE802(Native Method)
at org.jnetpcap.protocol.lan.Ethernet.calculateChecksum(Unknown Source)
at org.logstashplugins.RawUdpPacketSender.sendPacket(RawUdpPacketSender.java:156)
at org.logstashplugins.RawUdpPacketSender.sendPacket(RawUdpPacketSender.java:52)
at org.logstashplugins.Spoof.output(Spoof.java:108)
at org.logstash.config.ir.compiler.JavaOutputDelegatorExt.outputRubyEvents(JavaOutputDelegatorExt.java:72)
at org.logstash.config.ir.compiler.JavaOutputDelegatorExt.doOutput(JavaOutputDelegatorExt.java:95)
at org.logstash.config.ir.compiler.AbstractOutputDelegatorExt.multiReceive(AbstractOutputDelegatorExt.java:101)
at org.logstash.generated.CompiledDataset30.compute(Unknown Source)
at org.logstash.generated.CompiledDataset33.compute(Unknown Source)
at org.logstash.execution.WorkerLoop.run(WorkerLoop.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:425)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:292)
at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:28)
at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:90)
at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)
at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_workers$2(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:239)
at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)
at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)
at org.jruby.runtime.Block.call(Block.java:124)
at org.jruby.RubyProc.call(RubyProc.java:295)
at org.jruby.RubyProc.call(RubyProc.java:274)
at org.jruby.RubyProc.call(RubyProc.java:270)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
Therefore that’s why this paragraph is amazing. Thanks!
]]>Interesting, I am still unable to replicate this issue, I have tried to send your message as well as empty messages. Funnily enough I am sending PaloAlto logs from our systems at thousands a second and have not encountered that error even after a month. I have updated the binary to print the message on send error to isolate the failed message. If you can please re-install and once encountering the message again provide a sample of the same message https://github.com/xucito/logstash-output-spoof/releases/tag/0.1.4 . You can also email me the message at [email protected]
]]>Ok thanks for update, I will test this today and investigate why this is happening.
]]>So, an update.
0.1.3 installed on our test platform also generates the UDP buffer underflow message.
0.1.0 installed on our production platform does not show the message.
I made no changes to the logstash configs, just stopped logstash, uninstalled one version and installed the other, and restarted logstash again
FYI, here is a typical message.
1 2020-06-11T11:40:20+12:00 AD-ENT-PA01.AD-ENT-PA01 – – – – 1,2020/06/11 11:40:20,016201010404,TRAFFIC,end,2049,2020/06/11 11:40:20,10.2.2.2,1.1.1.1,103.1.1.1,1.1.1.1,WWW2-BYOD,,,dns,vsys1,INTERNAL-TRUSTED,EXTERNAL-UNTRUSTED,ethernet1/3.2,ethernet1/1,SIEM01-QRADAR-SYSLOG-SRVR,2020/06/11 11:40:20,904774,1,59590,53,44027,53,0×400019,udp,allow,310,156,154,3,2020/06/11 11:39:50,0,any,0,1647120274,0×0,10.0.0.0-10.255.255.255,Australia,0,2,1,aged-out,103,179,0,0,,AD-ENT-PA01,from-policy,,,0,,0,,N/A,0,0,0,0
]]>I’m receiving a syslog message from a PaloAlto firewall into our logstash instance. First thing I do is copy the “message” content to a temporary variable. Then later on I try and use spoof to resend the original content to the same host but different port, so that I can get the filebeat built in PaloAlto parser to look at it. This worked ok on our test box using 0.1.0, but seems to be failing on our production box using 0.1.3. I will install 0.1.3 on our test box to see if the problem is reproduceable. If so I will remove 0.1.3 from production and try 0.1.0 in production.
]]>hm, any idea what type of message you are selling to generate this error message?
]]>Hi there
0.1.3 seems to be generating errors:
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: java.nio.BufferUnderflowException
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jnetpcap.util.checksum.Checksum.crc32IEEE802(Native Method)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jnetpcap.protocol.lan.Ethernet.calculateChecksum(Unknown Source)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstashplugins.RawUdpPacketSender.sendPacket(RawUdpPacketSender.java:156)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstashplugins.RawUdpPacketSender.sendPacket(RawUdpPacketSender.java:52)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstashplugins.Spoof.output(Spoof.java:107)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstash.config.ir.compiler.JavaOutputDelegatorExt.outputRubyEvents(JavaOutputDelegatorExt.java:92)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstash.config.ir.compiler.JavaOutputDelegatorExt.doOutput(JavaOutputDelegatorExt.java:115)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstash.config.ir.compiler.AbstractOutputDelegatorExt.multiReceive(AbstractOutputDelegatorExt.java:121)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstash.generated.CompiledDataset605.compute(Unknown Source)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.compute(CompiledPipeline.java:356)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstash.config.ir.CompiledPipeline$CompiledUnorderedExecution.compute(CompiledPipeline.java:346)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.logstash.execution.WorkerLoop.run(WorkerLoop.java:82)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at java.lang.reflect.Method.invoke(Method.java:498)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:441)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:305)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:32)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_workers$5(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:279)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jruby.runtime.Block.call(Block.java:139)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jruby.RubyProc.call(RubyProc.java:318)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
Jun 10 02:26:14 ah-noc-logstash01 logstash[14540]: at java.lang.Thread.run(Thread.java:748)