Discussion:
flux
clay teahouse
2015-05-26 11:35:27 UTC
Permalink
Hi All,

I am trying to test flux module for writing template driven topologies. I
setting the topologies in localcluster mode.
1) using template simple_wordcount.yaml
2) using kafka_spout.yaml (using TestBolt)

With (1) I don't get any output form TestBolt and the topology exits.
With (2), I get the following error:

java.lang.IllegalArgumentException: Couldn't find a suitable constructor
for class 'storm.kafka.KafkaConfig' with arguments
'[***@17d1e01f, TOPIC, /storm, myKafkaClientId]\


Any idea what I could be doing wrong?


thanks

Clay
P. Taylor Goetz
2015-05-26 15:49:05 UTC
Permalink
Hi Clay,

1) Are you overriding the default sleep time for local topologies with the `-s` option followed by the sleep time in milliseconds? If so, with what value? By default, flux will run local mode topologies for 60 seconds.

2) It looks like you are trying to use KafkaConfig with constructor arguments for SpoutConfig. The SpoutConfig class extends KafkaConfig and has a different constructor signature:

public KafkaConfig(BrokerHosts hosts, String topic)
public KafkaConfig(BrokerHosts hosts, String topic, String clientId)
public SpoutConfig(BrokerHosts hosts, String topic, String zkRoot, String id)

-Taylor
Post by clay teahouse
Hi All,
I am trying to test flux module for writing template driven topologies. I setting the topologies in localcluster mode.
1) using template simple_wordcount.yaml
2) using kafka_spout.yaml (using TestBolt)
With (1) I don't get any output form TestBolt and the topology exits.
Any idea what I could be doing wrong?
thanks
Clay
clay teahouse
2015-05-26 16:05:16 UTC
Permalink
Hello Taylor,
1) I am not overriding anything.
2) You are right. I did notice the mistake and fixed it, but, now I am in
the same situation as in (1).

thanks,
Clay
Post by P. Taylor Goetz
Hi Clay,
1) Are you overriding the default sleep time for local topologies with the
`-s` option followed by the sleep time in milliseconds? If so, with what
value? By default, flux will run local mode topologies for 60 seconds.
2) It looks like you are trying to use KafkaConfig with constructor
arguments for SpoutConfig. The SpoutConfig class extends KafkaConfig and
public KafkaConfig(BrokerHosts hosts, String topic)
public KafkaConfig(BrokerHosts hosts, String topic, String clientId)
public SpoutConfig(BrokerHosts hosts, String topic, String zkRoot, String id)
-Taylor
Hi All,
I am trying to test flux module for writing template driven topologies. I
setting the topologies in localcluster mode.
1) using template simple_wordcount.yaml
2) using kafka_spout.yaml (using TestBolt)
With (1) I don't get any output form TestBolt and the topology exits.
java.lang.IllegalArgumentException: Couldn't find a suitable constructor
for class 'storm.kafka.KafkaConfig' with arguments
Any idea what I could be doing wrong?
thanks
Clay
P. Taylor Goetz
2015-05-26 16:53:49 UTC
Permalink
I just tested with the following command (from the “flux-examples” directory):

storm jar target/flux-examples-0.3.1-SNAPSHOT.jar org.apache.storm.flux.Flux ./src/main/resources/simple_wordcount.yaml

and it ran fine, logging the running count totals at the INFO level:

95957 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [nathan, 100]
96057 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [golda, 90]
96157 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [jackson, 90]
96259 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [nathan, 101]
96360 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [jackson, 91]
96461 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [jackson, 92]
96562 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [jackson, 93]
96663 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [mike, 113]
96763 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [nathan, 102]

Can you post the output you are seeing where it just exists?

-Taylor
Post by clay teahouse
Hello Taylor,
1) I am not overriding anything.
2) You are right. I did notice the mistake and fixed it, but, now I am in the same situation as in (1).
thanks,
Clay
Hi Clay,
1) Are you overriding the default sleep time for local topologies with the `-s` option followed by the sleep time in milliseconds? If so, with what value? By default, flux will run local mode topologies for 60 seconds.
public KafkaConfig(BrokerHosts hosts, String topic)
public KafkaConfig(BrokerHosts hosts, String topic, String clientId)
public SpoutConfig(BrokerHosts hosts, String topic, String zkRoot, String id)
-Taylor
Post by clay teahouse
Hi All,
I am trying to test flux module for writing template driven topologies. I setting the topologies in localcluster mode.
1) using template simple_wordcount.yaml
2) using kafka_spout.yaml (using TestBolt)
With (1) I don't get any output form TestBolt and the topology exits.
Any idea what I could be doing wrong?
thanks
Clay
clay teahouse
2015-05-26 19:42:24 UTC
Permalink
Taylor,

I don't see any output, except the message from zookeeper about the session
termination. I must be missing something on my side. I'll look around to
see if I can find anything.

thank you
Clay
I just tested with the following command (from the “flux-examples”
storm jar target/flux-examples-0.3.1-SNAPSHOT.jar
org.apache.storm.flux.Flux ./src/main/resources/simple_wordcount.yaml
95957 [Thread-11-bolt-2] INFO
org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2,
stream: default, id: {}, [nathan, 100]
96057 [Thread-11-bolt-2] INFO
org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2,
stream: default, id: {}, [golda, 90]
96157 [Thread-11-bolt-2] INFO
org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2,
stream: default, id: {}, [jackson, 90]
96259 [Thread-11-bolt-2] INFO
org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2,
stream: default, id: {}, [nathan, 101]
96360 [Thread-11-bolt-2] INFO
org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2,
stream: default, id: {}, [jackson, 91]
96461 [Thread-11-bolt-2] INFO
org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2,
stream: default, id: {}, [jackson, 92]
96562 [Thread-11-bolt-2] INFO
org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2,
stream: default, id: {}, [jackson, 93]
96663 [Thread-11-bolt-2] INFO
org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2,
stream: default, id: {}, [mike, 113]
96763 [Thread-11-bolt-2] INFO
org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2,
stream: default, id: {}, [nathan, 102]
Can you post the output you are seeing where it just exists?
-Taylor
Hello Taylor,
1) I am not overriding anything.
2) You are right. I did notice the mistake and fixed it, but, now I am in
the same situation as in (1).
thanks,
Clay
Post by P. Taylor Goetz
Hi Clay,
1) Are you overriding the default sleep time for local topologies with
the `-s` option followed by the sleep time in milliseconds? If so, with
what value? By default, flux will run local mode topologies for 60 seconds.
2) It looks like you are trying to use KafkaConfig with constructor
arguments for SpoutConfig. The SpoutConfig class extends KafkaConfig and
public KafkaConfig(BrokerHosts hosts, String topic)
public KafkaConfig(BrokerHosts hosts, String topic, String clientId)
public SpoutConfig(BrokerHosts hosts, String topic, String zkRoot, String id)
-Taylor
Hi All,
I am trying to test flux module for writing template driven topologies. I
setting the topologies in localcluster mode.
1) using template simple_wordcount.yaml
2) using kafka_spout.yaml (using TestBolt)
With (1) I don't get any output form TestBolt and the topology exits.
java.lang.IllegalArgumentException: Couldn't find a suitable constructor
for class 'storm.kafka.KafkaConfig' with arguments
Any idea what I could be doing wrong?
thanks
Clay
P. Taylor Goetz
2015-05-26 22:09:36 UTC
Permalink
Sounds like you might have a logback config that set the default level to WARN...

Can you check your configuration for that?

-Taylor
Post by clay teahouse
Taylor,
I don't see any output, except the message from zookeeper about the session termination. I must be missing something on my side. I'll look around to see if I can find anything.
thank you
Clay
Post by P. Taylor Goetz
storm jar target/flux-examples-0.3.1-SNAPSHOT.jar org.apache.storm.flux.Flux ./src/main/resources/simple_wordcount.yaml
95957 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [nathan, 100]
96057 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [golda, 90]
96157 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [jackson, 90]
96259 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [nathan, 101]
96360 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [jackson, 91]
96461 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [jackson, 92]
96562 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [jackson, 93]
96663 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [mike, 113]
96763 [Thread-11-bolt-2] INFO org.apache.storm.flux.wrappers.bolts.LogInfoBolt - source: bolt-1:2, stream: default, id: {}, [nathan, 102]
Can you post the output you are seeing where it just exists?
-Taylor
Post by clay teahouse
Hello Taylor,
1) I am not overriding anything.
2) You are right. I did notice the mistake and fixed it, but, now I am in the same situation as in (1).
thanks,
Clay
Post by P. Taylor Goetz
Hi Clay,
1) Are you overriding the default sleep time for local topologies with the `-s` option followed by the sleep time in milliseconds? If so, with what value? By default, flux will run local mode topologies for 60 seconds.
public KafkaConfig(BrokerHosts hosts, String topic)
public KafkaConfig(BrokerHosts hosts, String topic, String clientId)
public SpoutConfig(BrokerHosts hosts, String topic, String zkRoot, String id)
-Taylor
Post by clay teahouse
Hi All,
I am trying to test flux module for writing template driven topologies. I setting the topologies in localcluster mode.
1) using template simple_wordcount.yaml
2) using kafka_spout.yaml (using TestBolt)
With (1) I don't get any output form TestBolt and the topology exits.
Any idea what I could be doing wrong?
thanks
Clay
Loading...