JPA Databases micro-integration plugin problems

Hello,

As this is my first post - welcome to the group .

We are struggling with running https://solace.com/integration-hub/databases-jpa/ - (version 2.0.1) micro integration plugin. I did not have any problems with running other plugins like transformation etc. so I think I got the concept of those plugins correctly.

Trying to run the solace example /samples/config/application.yml ends up with :

The spring.config.import property is missing a configserver: entry

I’ve managed to overcome this issue with adding

    config:
      import-check:
        enabled: false

but then we got :

Required key 'spring.cloud.function.definition' not found

I’ve also added this property with some dummy empty entry :

  cloud:
    function:
      definition:

then :

management.datadog.metrics.export.apiKey was 'null' but it is required

added :


  datadog:
    metrics:
      export:
        enabled: false

And finally, we come to an error which im not able to overcome :

org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiateWithFactoryMethod(SimpleInstantiationStrategy.java:88)
 
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:168)
 
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
 
... 21 common frames omitted
 
Caused by: java.lang.IllegalArgumentException: No binder found for binding input-0

At first it looks like some binder relation is wrong, but I;ve tried almost everything and cannot understand why this cannot find the solace.java binder to the broker. We did not have such problems with different plugins. I’ve tried also the multi-system syntax giving the “binders” definition but this also did not help.

Looking through the logs it looks like problem occurse after connecting to database, but before connecting to solace broker.

The current anonymized yaml looks like :

# --------------------------
# SAMPLE CONFIGURATION
# --------------------------

spring:
  jpa:
    hibernate:
      ddl-auto: none
  config:
    import: application-db-processor.yml
  boot:
    admin:
      client:
        enabled: true
        url: http://localhost:8082
        auto-registration: true
        instance:
          name: DataBase(Source)
          service-url: http://localhost:${server.port}
    application:
      name: DataBase(Sink)
  cloud:

    function:
      definition:

    # For more info about Spring Cloud Stream and the Solace PubSub+ binder:
    # - https://docs.spring.io/spring-cloud-stream/docs/current/reference/html/spring-cloud-stream.html
    # - https://github.com/SolaceProducts/solace-spring-cloud/tree/master/solace-spring-cloud-starters/solace-spring-cloud-stream-starter
    stream:
      bindings:  # Workflow bindings
        input-0:
          destination: test.pass.q  # Queue name
          binder: solace
          consumer:
            #Concurrency can be used with a consumer group (non-exclusive queue) to process events in multiple threads
            batch-mode: true
            concurrency: 1
            max-attempts: 2
            back-off-initial-interval: 5000
            back-off-multiplier: 1
            default-retryable: true
            retryable-exceptions:
              org.springframework.orm.jpa.JpaSystemException: true
        output-0:
          destination: test/passenger/lvq   # Database producer destination
          binder: solace-db
        input-1:
          destination: test.addr.q  # Queue name
          binder: solace
          consumer:
            #Concurrency can be used with a consumer group (non-exclusive queue) to process events in multiple threads
            batch-mode: true
            concurrency: 1
            max-attempts: 2
            back-off-initial-interval: 5000
            back-off-multiplier: 1
            default-retryable: true
            retryable-exceptions:
              org.springframework.orm.jpa.JpaSystemException: true
        output-1:
          destination: test/addr/lvq   # Database producer destination
          binder: solace-db
      solace:
        bindings:
          input-0:
            consumer:
              batchMaxSize: 5
              batchTimeout: 5000
              provisionSubscriptionsToDurableQueue: false
              provisionDurableQueue: false
              queueNamePrefix: ""
              useFamiliarityInQueueName: false
              useDestinationEncodingInQueueName: false
              useGroupNameInQueueName: false
          input-1:
            consumer:
              batchMaxSize: 5
              batchTimeout: 5000
              provisionSubscriptionsToDurableQueue: false
              provisionDurableQueue: false
              queueNamePrefix: ""
              useFamiliarityInQueueName: false
              useDestinationEncodingInQueueName: false
              useGroupNameInQueueName: false
      #      binding-retry-interval: 30  # The interval (in seconds) between retrying binding creation. Set it to zero to treat such conditions as fatal, preventing the application from starting.

    config:
      import-check:
        enabled: false


solace:
  connector:
    workflows:  # Workflow configuration
      0:
        enabled: true  # If true, the workflow is enabled.
      1:
        enabled: false  # If true, the workflow is enabled.

    #        transform-headers:  # Per-Processor headers transform configuration
    #          expressions:  # A mapping of header names to header value SpEL expressions. The SpEL context contains the `headers` parameter which can be used to read the input message’s headers.
    #            new-header: "'prefix-' + headers.id"  # Example
    #        transform-payloads:  # Per-Processor payloads transform configuration
    #          expressions:  # A list of a single transformation as a SpEL expression
    #            - transform: "#isPayloadBytes(payload) ? new String(payload).toUpperCase() : payload instanceof T(String) ? payload.toUpperCase() : payload"  # Example

    management:
      leader-election:
        mode: standalone  # The connector’s leader election mode. (values: standalone, active_active, active_standby)
        fail-over:
          max-attempts: 3  # The maximum number of attempts to perform a fail-over.
          back-off-initial-interval: 1000  # The initial interval (milliseconds) to back-off when retrying a fail-over.
          back-off-max-interval: 10000  # The maximum interval (milliseconds) to back-off when retrying a fail-over.
          back-off-multiplier: 2.0  # The multiplier to apply to the back-off interval between each retry of a fail-over.
      queue: management-queue-push-pass  # The management queue name.
      session:  # The management session. This has the same interface as that used by `solace.java.*`. For more info: https://github.com/SolaceProducts/solace-spring-boot/tree/master/solace-spring-boot-starters/solace-java-spring-boot-starter#updating-your-application-properties
        host: tcp://localhost:55555
        msgVpn: default
        clientUsername: admin
        clientPassword: admin

    security:
      enabled: false  # If true, security is enabled. Otherwise, anyone has access to the connector’s endpoints.
  #      csrf-enabled: true  # If true, CSRF protection is enabled. Makes sense only if solace.connector.security.enabled is true.
  #      users:  # User configuration
  #        - name: <user-name>  # The name of this user.
  #          password: <user-password>  # The password for this user.
  #          roles:  # The list of roles which this user has. Has read-only access if no roles are given. (values: admin)
  #            - <role>

  #        - name: user  # Example: Creates a read-only user
  #          password: pass
  #        - name: admin  # Example: Creates a user with admin role
  #          password: admin
  #          roles:
  #            - admin


  java:  # Solace PubSub+ connection details. For more info: https://github.com/SolaceProducts/solace-spring-boot/tree/master/solace-spring-boot-starters/solace-java-spring-boot-starter#updating-your-application-properties
    host: tcp://host:55555
    msgVpn: default
    clientUsername: admin
    clientPassword: admin
    connect-retries: -1
    reconnect-retries: -1

management:
  metrics:  # Metrics monitoring systems. For more info: https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuator.metrics
    export:
      #      defaults:
      #        enabled: false
      simple:
        enabled: true
  #      <system>:
  #        enabled: false
  endpoint:
    #    health:
    #      show-components: never
    #      show-details: never
    logfile:
      external-file: log/connector/runtime.log

  datadog:
    metrics:
      export:
        enabled: false

        
  endpoints:
    jmx:
      exposure:
        exclude: "*"
    web:  # Actuator web endpoint configuration. For more info: https://docs.spring.io/spring-boot/docs/current/reference/html/actuator.html#actuator.endpoints. Initial set of metrics is defined as a bare minimum for Connector Manager compatibility. To shrink it simply remove unnecessary ones
      exposure:
        include: "health,metrics,loggers,logfile,channels,env,workflows,leaderelection,bindings"
        exclude: "diskSpace,mappings"

#logging:  # Logging configuration. For more info: https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.logging
#  level:
#    root: warn
#    com.solace: TRACE # Enables TRACE logs for connector
#    com.solace.connector.core: TRACE # Enables TRACE logs for connector framework
#    com.solace.spring.cloud.stream.binder: TRACE # Enables TRACE logs for Solace binder
#  file:
#    name: myapp.log  # base log file name. Example: Generates a log file to a file named myapp.log.
#  logback:
#    rollingpolicy:  # Logback file rolling policy
#      file-name-pattern: "${LOG_FILE}.%d{yyyy-MM-dd}.%i.gz"  # Pattern for rolled-over log file names. Example: Rolls the log file into an archive once every day. `%i` is reset to `0` at the start of the day.
#      max-file-size: 100MB  # The maximum size of log file before it is archived. Example: Within a given day, when the log file reaches 100 MB, archive it, and increment `%i`.
#      max-history: 7  # The maximum rotation-period's worth of archive log files to keep. Example: Since the `file-name-pattern` is configured to rollover once every day, keep 7 days worth of log archives.
#      total-size-cap: 1GB  # Total size of log backups for a given rotation period. Example: Keep up-to 1 GB of log archives for every day.

server:
  port: 8090
logging:
  level:
    root: INFO
    com.solacesystems.jcsmp: INFO
    com.solace.spring.cloud.stream.binders.db: INFO
    com.solace.spring: ERROR
    org.springframework: ERROR
    org.springframework.context.support.DefaultLifecycleProcessor: ERROR
    org.hibernate.SQL: ERROR
    org.hibernate.type.descriptor.sql.BasicBinder: ERROR
    com.solace.spring.cloud.stream.binders.db.jdbc.DatabaseUtil: ERROR

jasypt:
  encryptor:
    password: dummy
    algorithm: PBEWithMD5AndDES
    iv-generator-classname: org.jasypt.iv.NoIvGenerator

solace-persistence:
  datasource:
    #    driver-class-name: com.mysql.cj.jdbc.Driver
    #    username: root
    #    url: jdbc:mysql://localhost:3306/dtstest
    #    url: jdbc:mysql://10.144.23.240:3306/dtstest
    #    password: rootroot
    driver-class-name: com.sap.db.jdbc.Driver
    url: jdbc:sap://local:443?databaseName=H00&encrypt=true
    username: anonym
    password: gall

#    driver-class-name: com.microsoft.sqlserver.jdbc.SQLServerDriver
#    url: jdbc:sqlserver://localhost:1433;databaseName=test;Encrypt=false
#    username: sa
#    password: MSqwerasdf@!
    #    driver-class-name: com.vertica.jdbc.Driver
    #    url: jdbc:vertica://ec2-3-91-97-41.compute-1.amazonaws.com:5433/VMart
    #    username: dbadmin
    #    password:

    hikari:
      minimum-idle: 1
      maximum-pool-size: 5
      idle-timeout: 500000
      max-lifetime: 540000
      connection-timeout: 5000
  jpa:
    hibernate:
      ddl-auto: update
    show-sql: true
    properties:
      hibernate:
        jdbc:
          batch_size: 100
        order_updates: true
        order_inserts: true
        generate_statistics: false
#        dialect: org.hibernate.dialect.Oracle10gDialect
        #dialect: org.hibernate.dialect.DB2Dialect
        #dialect: org.hibernate.dialect.SQLServerDialect
        #        dialect: org.hibernate.dialect.PostgreSQL10Dialect
        #        default_schema: public   #POSTGRESQL specific
        #dialect: org.hibernate.dialect.VerticaDialect
        #dialect: org.hibernate.dialect.MySQLDialect
        #dialect: org.hibernate.dialect.HANARowStoreDialect
    #database: mysql
    #database: hana
    #database: db2
    #database: sqlserver
  #    database: POSTGRESQL
  sink:
    db-crud-mode: JPA
    jpaBatchMode: true
    jpaBatchSize: 60
    jpaMaxRetry: 2
    jpaRetryWaitMilliseconds:  3000
    redirectOnFailEnable: false
    redirectPrefix: RD/
    payloadFormat: json


solace-db:
  bindings:
    output-0:
      producer:
        entities:
          MasterdataapiDbStorecontract:
            clazz: MasterdataapiDbStorecontract

maybe im missing something obvious
appleciate any tips, hits -

Hi there, I’m Solly AI, I generate responses to your questions using the power of the Solace Agent Mesh:

Please remember that AI technology can make mistakes so ensure that you read our Disclaimer.

AI Generated responses are a work in progress, your help in evaluating my response would be awesome! If my response makes sense, please mark it as a solution or give a :+1:. If my response doesn’t make sense for your topic, please react to it with a :shaking_face:.

Looking at the documentation, I don’t see any specific information about resolving issues with the database integration plugin for a software broker. However, I can provide some general troubleshooting guidance based on the available information:

  1. First, verify that your broker deployment meets the minimum resource requirements as specified in the System Resource Calculator. Insufficient resources can cause plugin issues.

  2. Check that the storage configuration is properly set up, as plugins require access to the storage-group directory mounted at /var/lib/solace.

  3. Review the broker logs located in /var/lib/solace/jail/logs for any error messages related to the plugin.

  4. Consider gathering diagnostics using the gather-diagnostics script which can help identify issues.

For more detailed troubleshooting, you may want to check these resources:

Since there isn’t specific documentation about database integration plugin issues, I recommend:

  1. Checking the plugin documentation for specific requirements
  2. Reviewing the broker logs for plugin-specific errors
  3. Verifying that all plugin dependencies are met
  4. Ensuring proper plugin configuration in the broker