使用logstash将csv文件传输到服务器

我目前正在学习使用大数据, 客户端:Windows 10 服务器端:CentOS7 elasticsearch,logstash和kibana已安装在服务器端, 作为客户端,我可以通过http://10.64.2.246:5601/访问kibana,一切都很酷。现在,我想学习如何将数据从我的PC传输到服务器端的elasticsearch。我已经在客户端下载了logstash。

我已经用2列从kaggle准备了数据。 我已经制作了我的conf文件

conf文件

input   {
    file    {
        path => "C:\Users\Charles\Desktop\DATA"
        start_position => "beginning"
        sincedb_path=>"/dev/null"
    }
}
filter  {
    csv     {
        separator=>","
        columns=>["Date","Price"]
    }
    mutate {convert =>["Price","float"]}
}
output{
    elasticsearch{
        host => "10.64.2.246"
        index=>"oil"
        document_type=>"minyaks"
    }
    stdout{}
}

这是我执行在客户端执行的conf文件的代码。

bin/logstash -f /Users/Charles/Desktop/DATA/logstash_oil.conf

这是显示的错误

Thread.exclusive is deprecated,use Thread::Mutex
Sending Logstash logs to C:/Users/Charles/Desktop/DATA/logstash-7.4.2/logs which is now configured via log4j2.properties
[2019-11-07T20:33:11,077][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-11-07T20:33:11,090][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.4.2"}
[2019-11-07T20:33:12,416][INFO ][org.reflections.Reflections] Reflections took 29 ms to scan 1 urls,producing 20 keys and 40 values
[2019-11-07T20:33:12,917][ERROR][logstash.outputs.elasticsearch] Unknown setting 'host' for elasticsearch
[2019-11-07T20:33:12,925][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::Pipelineaction::Create/pipeline_id:main,:exception=>"Java::JavaLang::IllegalStateException",:message=>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.",:backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:100)","org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:60)","org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)","org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)","org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1156)","org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1143)","org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)","C_3a_.Users.Charles.Desktop.DATA.logstash_minus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(C:/Users/Charles/Desktop/DATA/logstash-7.4.2/logstash-core/lib/logstash/java_pipeline.rb:26)","org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)","org.jruby.internal.runtime.methods.MixedmodeIRMethod.call(MixedmodeIRMethod.java:90)","org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)","org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)","org.jruby.RubyClass.newInstance(RubyClass.java:915)","org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)","org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)","C_3a_.Users.Charles.Desktop.DATA.logstash_minus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(C:/Users/Charles/Desktop/DATA/logstash-7.4.2/logstash-core/lib/logstash/pipeline_action/create.rb:36)","C_3a_.Users.Charles.Desktop.DATA.logstash_minus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(C:/Users/Charles/Desktop/DATA/logstash-7.4.2/logstash-core/lib/logstash/pipeline_action/create.rb)","C_3a_.Users.Charles.Desktop.DATA.logstash_minus_7_dot_4_dot_2.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(C:/Users/Charles/Desktop/DATA/logstash-7.4.2/logstash-core/lib/logstash/agent.rb:326)","org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)","org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)","org.jruby.runtime.Block.call(Block.java:129)","org.jruby.RubyProc.call(RubyProc.java:295)","org.jruby.RubyProc.call(RubyProc.java:274)","org.jruby.RubyProc.call(RubyProc.java:270)","org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)","java.lang.Thread.run(Unknown Source)"]}
warning: thread "Converge Pipelineaction::Create<main>" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `Pipelineaction::Create<main>`
          create at org/logstash/execution/ConvergeResultExt.java:109
             add at org/logstash/execution/ConvergeResultExt.java:37
  converge_state at C:/Users/Charles/Desktop/DATA/logstash-7.4.2/logstash-core/lib/logstash/agent.rb:339
[2019-11-07T20:33:12,932][ERROR][logstash.agent           ] An exception happened when converging configuration {:exception=>LogStash::Error,:message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `Pipelineaction::Create<main>`",:backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:in `create'","org/logstash/execution/ConvergeResultExt.java:37:in `add'","C:/Users/Charles/Desktop/DATA/logstash-7.4.2/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]}
[2019-11-07T20:33:12,954][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `Pipelineaction::Create<main>`>,"C:/Users/Charles/Desktop/DATA/logstash-7.4.2/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]}
[2019-11-07T20:33:13,004][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

需要某人向我解释。我遵循youtube上的指示,但是讲师是在本地进行的。

谢谢。

FANCY905269369 回答:使用logstash将csv文件传输到服务器

问题出在这条线上:

[2019-11-07T20:33:12,917][ERROR][logstash.outputs.elasticsearch] Unknown setting 'host' for elasticsearch

所以您需要更改

    host => "10.64.2.246"

    hosts => "10.64.2.246"

此外,由于您似乎正在使用ES 7.4.2,因此可以将以下行删除为document/mapping types are deprecated

document_type=>"minyaks"
本文链接:https://www.f2er.com/3140267.html

大家都在问