I am currently using Kafka Connect version 6.1.1 and attempting to utilize Sergey34's kafka-connect-transformers library to modify my Kafka messages before sending them to BigQuery via the BigQuerySink.
Within my connector.properties file, I have set up the ScriptEngineTransformer configuration in the following manner (example shortened for clarity):
transforms=scriptenginetransformer
transforms.scriptenginetransformer.type=seko.kafka.connect.transformer.script.ScriptEngineTransformer
transforms.scriptenginetransformer.scrip_engine_name=javascript
transforms.scriptenginetransformer.value.script=function valueTransform(source){ source.foo = 42; return source;}
During runtime, I encounter the error message:
javax.script.ScriptException: TypeError: Cannot set property "foo" of Struct{a=111,b=222} in <eval> at line number 2
(full stack trace can be found here)
Based on my basic understanding of JavaScript, it seems like modifying or adding properties to a struct should be possible, as demonstrated in this simple sandbox example:
function Foo(x, y) {
this.x = x;
this.y = y;
}
foo = new Foo(1, 2);
foo.y = 3;
foo.z = 4;
console.log(foo);
The fact that my Avro-formatted Kafka message ({a=111,b=222}) is successfully passed to the JS script leads me to wonder what could be causing this particular error.