Wednesday, September 7, 2016

endeca dgraph nodes to navigation paramter mapping

Here are some sample mappings:
Web Application to APIAPI to MDEX Engine
/controller.jsp?N=0/graph?node=0
/controller.jsp?N=0&Ntk=DESC& Ntt=merlot/graph?node=0+attrs=DESC+merlot


MDEX Engine parameterDescriptionMaps to...
graph?Navigation queryN
nodeNavigation query parameter, navigation descriptorsN
offsetNavigation query parameter, record offsetNo
offsetNavigation query parameter, aggregated record offsetNao
groupNavigation query parameter, exposed refinementsNe
allbinsNavigation query parameter, records per aggregated recordNp
analyticsNavigation query parameter, analytics expression to apply to a queryNa
sortNavigation query parameter, sortNs
sortNavigation query parameter, sort orderNso
groupbyNavigation query parameter, rollupNu
attrsNavigation query parameter, record search key, terms, and optionsNtk, Ntt, Ntx
relrankNavigation query parameter, search interface, relevance ranking terms, relevance ranking strategy and match modeNrk, Nrt, Nrr, Nrm
dymNavigation query parameter, Did You MeanNty
autophraseNavigation query parameter, compute phrasingsNtpc
autophrasedwimNavigation query parameter, rewrite queryNtpr
merchpreviewtimeNavigation query parameter, merchandising preview timeNmpt
merchrulefilterNavigation query parameter, merchandising rule filterNmrf
predNavigation query parameter, range filtersNf
filterNavigation query parameter, record filtersNr
structuredNavigation query parameter, Endeca Query LanguageNrs
refinementNavigation query parameter, dynamic refinement rankingNrc
search?Dimension search queryD
termsDimension search query parameter, search termsD
optionsDimension search query parameter, optionsDx
nodeDimension search query parameter, dimension search scopeDn
modelDimension search query parameter, search dimensionDi
numDimension search query parameter, number of resultsDp
offsetDimension search query parameter, offsetDo
rankDimension search query parameter, rankDk
predDimension search query parameter, range filtersDf
filterDimension search query parameter, record filtersDr
structuredDimension search query parameter, Endeca Query LanguageDrs
abin?Aggregated record queryA
idAggregated record query parameter, record IDA
nodeAggregated record query parameter, descriptorsAn
groupbyAggregated record query parameter, rollupAu
predAggregated record query parameter, range filtersAf
filterAggregated record query parameter, record filtersAr
structuredAggregated record query parameter, Endeca Query LanguageArs
bin?Record queryR
idRecord query parameter, record IDR

Endeca Component 'Forge' is already started

There are instances when we run baseline update , it gets stuck at forge component indefinitely

$ ./baseline_update.sh

INFO: Checking definition from AppConfig.xml against existing EAC provisioning.
INFO: Definition has not changed.
INFO: Starting baseline update script.
INFO: Acquired lock 'update_lock'.
INFO: [ITLHost] Starting shell utility 'cleanDir_processing'.
INFO: [ITLHost] Starting shell utility 'cleanDir_forge-output'.
INFO: [ITLHost] Starting shell utility 'cleanDir_temp'.
INFO: [ITLHost] Starting shell utility 'move_-_to_processing'.
INFO: [ITLHost] Starting copy utility 'fetch_config_to_input_for_forge_Forge'.
INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_ConfigurationGeneratorForge'.
INFO: [ITLHost] Starting component 'ConfigurationGeneratorForge'.
INFO: [ITLHost] Starting backup utility 'backup_log_dir_for_component_Forge'.
INFO: [ITLHost] Starting component 'Forge'.
INFO: Component 'Forge' is already started.


If you run ps command , you will see 2 instances of Platformservices (EAC) running. The one with forge process is highlighted:

[<USER>@endeca control]$ ps -ef | grep java

<USER>       2130     1  2 Sep05 pts/0    01:02:33 <ENDECA-HOME>/PlatformServices/11.0.0/j2sdk/bin/java -Djava.util.logging.config.file=<ENDECA-HOME>/PlatformServices/workspace/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx1024m -XX:MaxPermSize=128m -Djava.security.auth.login.config=<ENDECA-HOME>/PlatformServices/workspace/conf/Login.conf -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,getLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,addLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,isGrantable -XX:+UnlockDiagnosticVMOptions -XX:-DisplayVMOutput -Dderby.stream.error.rollingFile.count=5 -Dderby.stream.error.rollingFile.limit=10240000 -Djava.endorsed.dirs=<ENDECA-HOME>/PlatformServices/11.0.0/tools/server/endorsed -classpath <ENDECA-HOME>/PlatformServices/11.0.0/tools/server/bin/bootstrap.jar:<ENDECA-HOME>/PlatformServices/11.0.0/tools/server/bin/tomcat-juli.jar -Dcatalina.base=<ENDECA-HOME>/PlatformServices/workspace -Dcatalina.home=<ENDECA-HOME>/PlatformServices/11.0.0/tools/server -Djava.io.tmpdir=<ENDECA-HOME>/PlatformServices/workspace/temp org.apache.catalina.startup.Bootstrap start
<USER>       2211     1  1 Sep05 pts/0    00:37:46 <ENDECA-HOME>/ToolsAndFrameworks/latest/server/j2sdk/bin/java -Djava.util.logging.config.file=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.security.auth.login.config=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace/conf/Login.conf -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,getLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,addLock -XX:CompileCommand=exclude,org/apache/derby/impl/services/locks/LockControl,isGrantable -XX:+UnlockDiagnosticVMOptions -XX:-DisplayVMOutput -d64 -Dderby.stream.error.rollingFile.count=5 -Dderby.stream.error.rollingFile.limit=10240000 -Xms256m -Xmx1024m -XX:MaxPermSize=512m -Dorg.apache.el.parser.SKIP_IDENTIFIER_CHECK=true -Dcom.endeca.manager.endecaroot=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace -Djava.endorsed.dirs=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/apache-tomcat-7.0.42/endorsed -classpath <ENDECA-HOME>/ToolsAndFrameworks/latest/server/apache-tomcat-7.0.42/bin/bootstrap.jar:<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace/bin/tomcat-juli.jar -Dcatalina.base=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace -Dcatalina.home=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/apache-tomcat-7.0.42 -Djava.io.tmpdir=<ENDECA-HOME>/ToolsAndFrameworks/latest/server/workspace/temp org.apache.catalina.startup.Bootstrap start
<USER>       2327  2324  0 Sep05 pts/0    00:03:45 <ENDECA-HOME>/CAS/latest/bin/../java/bin/java -Xms512m -Xmx2g -XX:MaxPermSize=128m -XX:+UseParallelOldGC -DSTOP.PORT=8506 -DSTOP.KEY=27931 -Dcom.endeca.cas.root=<ENDECA-HOME>/CAS/latest/bin/.. -Dcom.endeca.cas.workspace=<ENDECA-HOME>/CAS/latest/bin/../../workspace -Djetty.home=<ENDECA-HOME>/CAS/latest/bin/.. -Djava.io.tmpdir=<ENDECA-HOME>/CAS/latest/bin/../../workspace/temp -Dlog4j.configuration=file:<ENDECA-HOME>/CAS/latest/bin/../../workspace/conf/cas-service-log4j.properties -Dcom.endeca.itl.group.size=2048 -cp <ENDECA-HOME>/CAS/latest/bin/../../workspace/conf:<ENDECA-HOME>/CAS/latest/bin/../lib/server/*: com.endeca.itl.jetty.Main <ENDECA-HOME>/CAS/latest/bin/../../workspace/conf/jetty.xml
<USER>      15888  2130  2 17:31 pts/0    00:00:03 <ENDECA-HOME>/PlatformServices/11.0.0/bin/forge -vw --javaClasspath <ENDECA-HOME>/CAS/latest/lib/recordstore-forge-adapter/recordstore-forge-adapter-11.0.0.jar:./config/lib/java/spring-2.5.6.jar:./config/lib/java/eacToolkit-11.0.0.17592292.jar:./config/lib/java/spring-delegate-adapter-1.0.1.jar:./config/script:. --javaArgument -DENDECA_PROJECT_NAME=<APP-NAME> --javaArgument -Djava.util.logging.config.file=./config/script/logging.properties --logDir <ENDECA-HOME>/apps/<APP-NAME>/./logs/forges/Forge --outputPrefix <APP-NAME> --inputDir <ENDECA-HOME>/apps/<APP-NAME>/data/processing --outputDir <ENDECA-HOME>/apps/<APP-NAME>/data/forge_output --stateDir <ENDECA-HOME>/apps/<APP-NAME>/data/state --numPartitions 1 --tmpDir <ENDECA-HOME>/apps/<APP-NAME>/data/temp -o <ENDECA-HOME>/apps/<APP-NAME>/logs/forges/Forge/Forge.log <ENDECA-HOME>/apps/<APP-NAME>/data/processing/pipeline.epx


This happens when the baseline_update is stopped in between when forge process is running. The thread started by forge is still active . We will need to kill this process using unix kill command :

$ kill -9 15888

Baseline update should run without any issues now.