Endeca Component 'Forge' is already started

There are instances when we run baseline update , it gets stuck at forge component indefinitely

$ ./baseline_update.sh

INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
INFO: Definition has not changed.
INFO: Starting baseline update script.
INFO: Acquired lock 'update_lock'.
INFO: [ITLHost] Starting shell utility 'cleanDir_processing'.
INFO: [ITLHost] Starting shell utility 'cleanDir_forge-output'.
INFO: [ITLHost] Starting shell utility 'cleanDir_temp'.
INFO: [ITLHost] Starting shell utility 'move_-_to_processing'.
INFO: [ITLHost] Starting copy utility 'fetch_config_to_input_for_forge_Forge'.
INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_ConfigurationGeneratorForge'.
INFO: [ITLHost] Starting component 'ConfigurationGeneratorForge'.
INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_Forge'.
INFO: [ITLHost] Starting component 'Forge'.
INFO: Component 'Forge' is already started.


If you run ps command , you will see 2 instances of Platformservices (EAC) running. The one with forge process is highlighted:

[<USER>@endeca control]$ ps -ef | grep java

<USER>       2130     1  2 Sep05 pts/0    01:02:33 <ENDECA-HOME>/PlatformServices/11.0.0/j2sdk/bin/java -Djava.util.logging.config.file=<ENDECA-HOME>/PlatformServices/workspace/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -XX:MaxPermSize=128m -Djava.security.auth.login.config=<ENDECA-HOME>/PlatformServices/workspace/conf/Login.conf -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,getLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,addLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,isGrantable -XX:+UnlockDiagnosticVMOptions -XX:-DisplayVMOutput -Dderby.stream.error.rollingFile.count=5 -Dderby.stream.error.rollingFile.limit=10240000 -Djava.endorsed.dirs=<ENDECA-HOME>/PlatformServices/11.0.0/tools/server/endorsed -classpath <ENDECA-HOME>/PlatformServices/11.0.0/tools/server/bin/bootstrap.jar:<ENDECA-HOME>/PlatformServices/11.0.0/tools/server/bin/tomcat-juli.jar -Dcatalina.base=<ENDECA-HOME>/PlatformServices/workspace -Dcatalina.home=<ENDECA-HOME>/PlatformServices/11.0.0/tools/server -Djava.io.tmpdir=<ENDECA-HOME>/PlatformServices/workspace/temp org.apache.catalina.startup.Bootstrap start
<USER>       2211     1  1 Sep05 pts/0    00:37:46 <ENDECA-HOME>/ToolsAndFrameworks/latest/server/j2sdk/bin/java -Djava.util.logging.config.file=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.security.auth.login.config=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace/conf/Login.conf -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,getLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,addLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,isGrantable -XX:+UnlockDiagnosticVMOptions -XX:-DisplayVMOutput -d64 -Dderby.stream.error.rollingFile.count=5 -Dderby.stream.error.rollingFile.limit=10240000 -Xms256m -Xmx1024m -XX:MaxPermSize=512m -Dorg.apache.el.parser.SKIP_IDENTIFIER_CHECK=true -Dcom.endeca.manager.endecaroot=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace -Djava.endorsed.dirs=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/apache-tomcat-7.0.42/endorsed -classpath <ENDECA-HOME>/ToolsAndFrameworks/latest/server/apache-tomcat-7.0.42/bin/bootstrap.jar:<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace/bin/tomcat-juli.jar -Dcatalina.base=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace -Dcatalina.home=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/apache-tomcat-7.0.42 -Djava.io.tmpdir=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace/temp org.apache.catalina.startup.Bootstrap start
<USER>       2327  2324  0 Sep05 pts/0    00:03:45 <ENDECA-HOME>/CAS/latest/bin/../java/bin/java -Xms512m -Xmx2g -XX:MaxPermSize=128m -XX:+UseParallelOldGC -DSTOP.PORT=8506 -DSTOP.KEY=27931 -Dcom.endeca.cas.root=<ENDECA-HOME>/CAS/latest/bin/.. -Dcom.endeca.cas.workspace=<ENDECA-HOME>/CAS/latest/bin/../../workspace -Djetty.home=<ENDECA-HOME>/CAS/latest/bin/.. -Djava.io.tmpdir=<ENDECA-HOME>/CAS/latest/bin/../../workspace/temp -Dlog4j.configuration=file:<ENDECA-HOME>/CAS/latest/bin/../../workspace/conf/cas-service-log4j.properties -Dcom.endeca.itl.group.size=2048 -cp <ENDECA-HOME>/CAS/latest/bin/../../workspace/conf:<ENDECA-HOME>/CAS/latest/bin/../lib/server/*: com.endeca.itl.jetty.Main <ENDECA-HOME>/CAS/latest/bin/../../workspace/conf/jetty.xml
<USER>      15888  2130  2 17:31 pts/0    00:00:03 <ENDECA-HOME>/PlatformServices/11.0.0/bin/forge -vw --javaClasspath <ENDECA-HOME>/CAS/latest/lib/recordstore-forge-adapter/recordstore-forge-adapter-11.0.0.jar:./config/lib/java/spring-2.5.6.jar:./config/lib/java/eacToolkit-11.0.0.17592292.jar:./config/lib/java/spring-delegate-adapter-1.0.1.jar:./config/script:. --javaArgument -DENDECA_PROJECT_NAME=<APP-NAME> --javaArgument -Djava.util.logging.config.file=./config/script/logging.properties --logDir <ENDECA-HOME>/apps/<APP-NAME>/./logs/forges/Forge --outputPrefix <APP-NAME> --inputDir <ENDECA-HOME>/apps/<APP-NAME>/data/processing --outputDir <ENDECA-HOME>/apps/<APP-NAME>/data/forge_output --stateDir <ENDECA-HOME>/apps/<APP-NAME>/data/state --numPartitions 1 --tmpDir <ENDECA-HOME>/apps/<APP-NAME>/data/temp -o <ENDECA-HOME>/apps/<APP-NAME>/logs/forges/Forge/Forge.log <ENDECA-HOME>/apps/<APP-NAME>/data/processing/pipeline.epx


This happens when the baseline_update is stopped in between when forge process is running. The thread started by forge is still active . We will need to kill this process using unix kill command :

$ kill -9 15888

Baseline update should run without any issues now.