- Notifications
You must be signed in to change notification settings - Fork46
Description
Environment:
- Juman++ Version:
v2.0.0-rc4(from officialjumanpp-2.0.0-rc4.tar.xzrelease package) - OS: Windows Subsystem for Linux (WSL -
Ubuntu 24.04.2 LTS) - Compiler/Build Tools:
build-essential,cmake
Problem Description:
When attempting to use Juman++v2.0.0-rc4 built from the official release package, the program fails to load the providedjumandic.jppmdl model. The final error observed after a clean download and build is:failed to load model from disk: InvalidState: perceptron: failed to load perceptron information
Steps to Reproduce:
- Downloaded
jumanpp-2.0.0-rc4.tar.xz(verified size ~302MB). Most recent attempt involvedwgetdirectly into WSL to ensure file integrity. - Extracted the archive:
tar xf jumanpp-2.0.0-rc4.tar.xz - Navigated into
jumanpp-2.0.0-rc4. - Made the following modifications to successfully configure CMake:
- Created
version.cmakewith content:set(PROJECT_VERSION "2.0.0-rc4") - In
CMakeLists.txt:- Commented out
add_subdirectory(test) - Commented out
add_subdirectory(model)(within itsif (IS_DIRECTORY ...)block) - Added
install(DIRECTORY model/ DESTINATION libexec/jumanpp USE_SOURCE_PERMISSIONS)(e.g., afteradd_subdirectory(src))
- Commented out
- Created
- Created a build directory (
mkdir bld && cd bld). - Ran CMake:
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local - Compiled and installed:
sudo make install -j$(nproc) - Attempted to run Juman++:
echo "あ" | jumanpp --model=/usr/local/libexec/jumanpp/jumandic.jppmdl
Observed Behaviour:
- Initially, the model loading error was
InvalidState: perceptron: slice size was not equal to model size in header. - This was followed by a
Segmentation fault (core dumped)when Juman++ attempted to process input. - After a completely clean re-download of the
tar.xzpackage (usingwget), removal of old build artifacts and installed files, and re-application of the CMake fixes as detailed above, the error now consistently is:failed to load model from disk: InvalidState: perceptron: failed to load perceptron informationbacktrace: loadModel at /home/acomp/jumanpp-2.0.0-rc4/src/core/env.cc:18 init at /home/acomp/jumanpp-2.0.0-rc4/src/jumandic/shared/jumandic_env.cc:31
GDB Backtrace (from when the Segmentation Fault occurred in an earlier attempt):
The segmentation fault occurred before the "perceptron: failed to load perceptron information" error became the primary stopping point. This GDB backtrace points to an issue in the RNN component, likely due to the model not being loaded correctly.
Program received signal SIGSEGV, Segmentation fault. 0x000055555580b9b4 in jumanpp::rnn::mikolov::MikolovRnn::applyParallel(jumanpp::rnn::mikolov::ParallelStepData*) const () (gdb) bt #0 0x000055555580b9b4 in jumanpp::rnn::mikolov::MikolovRnn::applyParallel(jumanpp::rnn::mikolov::ParallelStepData*) const () #1 0x00005555556423d5 in jumanpp::core::analysis::GbeamRnnState::scoreBoundary(unsigned int) () #2 0x0000555555642c98 in jumanpp::core::analysis::GbeamRnnState::scoreLattice(jumanpp::core::analysis::Lattice*, jumanpp::core::analysis::ExtraNodesContext const*) () #3 0x000055555563eb5c in jumanpp::core::analysis::RnnScorerGbeam::scoreLattice(jumanpp::core::analysis::Lattice*, jumanpp::core::analysis::ExtraNodesContext const*, unsigned int) () #4 0x000055555562ee61 in jumanpp::core::analysis::AnalyzerImpl::computeScoresGbeam(jumanpp::core::analysis::ScorerDef const*) () #5 0x0000555555630781 in jumanpp::core::analysis::AnalyzerImpl::computeScores(jumanpp::core::analysis::ScorerDef const*) () #6 0x000055555562d21d in jumanpp::core::analysis::Analyzer::analyze(jumanpp::StringPiece, jumanpp::core::analysis::ScorePlugin*) () #7 0x00005555556832a7 in jumanpp::core::input::PlainStreamReader::analyzeWith(jumanpp::core::analysis::Analyzer*) () #8 0x00005555555a92d3 in main ()Expected Behaviour:
Juman++ should successfully load thejumandic.jppmdl model included in the release package and perform morphological analysis.
Suspicion:
This persistent failure related to the "perceptron" component of the model, even after ensuring a pristine download and build environment, suggests a potential issue with thejumandic.jppmdl file as distributed in thejumanpp-2.0.0-rc4.tar.xz package, or a bug in the Juman++ v2.0.0-rc4 code that prevents it from correctly parsing this model.