forked fromlowcoder-org/lowcoder
- Notifications
You must be signed in to change notification settings - Fork0
New plugin system#1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Uh oh!
There was an error while loading.Please reload this page.
Merged
Changes fromall commits
Commits
Show all changes
47 commits Select commitHold shift + click to select a range
12d7b8c WIP: intermediate changes
ludomikulad3cf4ba WIP: new plugin system first iteration
ludomikulaa119225 fix: check plugin endpoint method for proper return class
ludomikulaf42b4f5 new: make default max query timeout configurable
ludomikula005611d WIP: new plugin system first iteration
ludomikulaf3358b5 fix: check plugin endpoint method for proper return class
ludomikula1b820ca Add support for SUPER_ADMIN role
aq-ikhwa-techec518d8 Publish server log event for ee plugin to consume
aq-ikhwa-tech8b30ced new: reworked Plugin classloader system
ludomikula0b54e90 WIP: intermediate changes
ludomikula47d1dad WIP: new plugin system first iteration
ludomikula0451776 fix: check plugin endpoint method for proper return class
ludomikula042f511 new: make default max query timeout configurable
ludomikula0cfad7d WIP: new plugin system first iteration
ludomikulad8ed9c1 fix: check plugin endpoint method for proper return class
ludomikuladda057d Publish server log event for ee plugin to consume
aq-ikhwa-tech8d0c483 new: reworked Plugin classloader system
ludomikulae5afe8f Add handling for audit logs feature
aq-ikhwa-techa8cc6d9 Add handling for geolocation data for audit logs
aq-ikhwa-tech39e3e1f new: extend new plugin system
ludomikula4bd4373 WIP: intermediate changes
ludomikula6e57773 WIP: new plugin system first iteration
ludomikula1dff91e fix: check plugin endpoint method for proper return class
ludomikula04498f9 new: make default max query timeout configurable
ludomikula77ea427 WIP: new plugin system first iteration
ludomikulac60b4da fix: check plugin endpoint method for proper return class
ludomikula46179cb Publish server log event for ee plugin to consume
aq-ikhwa-techf81924f new: reworked Plugin classloader system
ludomikulae582b1e WIP: intermediate changes
ludomikulad394e76 WIP: new plugin system first iteration
ludomikula4429800 fix: check plugin endpoint method for proper return class
ludomikula3194009 new: make default max query timeout configurable
ludomikula25dad14 WIP: new plugin system first iteration
ludomikula67d6a74 fix: check plugin endpoint method for proper return class
ludomikulad1224a1 Publish server log event for ee plugin to consume
aq-ikhwa-tech4ae0e0a new: reworked Plugin classloader system
ludomikula4930575 Add handling for audit logs feature
aq-ikhwa-tech63d5dde Add handling for geolocation data for audit logs
aq-ikhwa-tech7a3b3e5 new: extend new plugin system
ludomikulaea9d6fc Add handling for api delays in case of rate limit
aq-ikhwa-tech083110c fix: reload router functions when a plugin adds endpoints
ludomikulac5f6d5d wip: plugin endpoint authentication
ludomikula19b0c4d new: propagate plugin specific environment variables to plugins
ludomikulaf145904 new: add environment variable for controlling plugin location
ludomikulacdc42c6 new: implemented plugin endpoints security
ludomikulaacc402f Merge remote-tracking branch 'origin/new_plugin_system' into merge-wi…
ludomikulad0b2b09 new: merge with latest LC
ludomikulaFile filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
There are no files selected for viewing
3 changes: 1 addition & 2 deletions.gitignore
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
24 changes: 15 additions & 9 deletionsdeploy/docker/Dockerfile
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
6 changes: 5 additions & 1 deletiondeploy/docker/api-service/entrypoint.sh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
6 changes: 3 additions & 3 deletionsserver/api-service/.gitignore
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
122 changes: 122 additions & 0 deletionsserver/api-service/PLUGIN.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| # Lowcoder plugin system (WIP) | ||
| This is an ongoing effort to refactor current plugin system based on pf4j library. | ||
| ## Reasoning | ||
| 1. create a cleaner and simpler plugin system with clearly defined purpose(s) (new endpoints, new datasource types, etc..) | ||
| 2. lowcoder does not need live plugin loading/reloading/unloading/updates, therefore the main feature of pf4j is rendered useless, in fact it adds a lot of complexity due to classloaders used for managing plugins (especially in spring/boot applications) | ||
| 3. simpler and easier plugin detection - just a jar with a class implementing a common interface (be it a simple pojo project or a complex spring/boot implementation) | ||
| ## How it works | ||
| The main entrypoint for plugin system is in **lowcoder-server** module with class **org.lowcoder.api.framework.configuration.PluginConfiguration** | ||
| It creates: | ||
| - LowcoderPluginManager bean which is responsible for plugin lifecycle management | ||
| - Adds plugin defined endpoints to lowcoder by creating **pluginEndpoints** bean | ||
| - TODO: Adds plugin defined datasources to lowcoder by creating **pluginDatasources** bean | ||
| ### lowcoder-plugin-api library | ||
| This library contains APIs for plugin implementations. | ||
| It is used by both, lowcoder API server as well as all plugins. | ||
| ### PluginLoader | ||
| The sole purpose of a PluginLoader is to find plugin candidates and load them into VM. | ||
| There is currently one implementation that based on paths - **PathBasedPluginLoader**, it: | ||
| - looks in folders and subfolders defined in **application.yaml** - entries can point to a folder or specific jar file. If a relative path is supplied, the location of lowcoder API server application jar is used as parent folder (when run in non-packaged state, eg. in IDE, it uses the folder where ServerApplication.class is generated) | ||
| ```yaml | ||
| common: | ||
| plugin-dirs: | ||
| - plugins | ||
| - /some/custom/path/myGreatPlugin.jar | ||
| ``` | ||
| - finds all **jar**(s) and inspects them for classes implementing **LowcoderPlugin** interface | ||
| - instantiates all LowcoderPlugin implementations | ||
| ### LowcoderPluginManager | ||
| The main job of plugin manager is to: | ||
| - register plugins found and instantiated by **PluginLoader** | ||
| - start registered plugins by calling **LowcoderPlugin.load()** method | ||
| - create and register **RouterFunction**(s) for all loaded plugin endpoints | ||
| - TODO: create and register datasources for all loaded plugin datasources | ||
| ## Plugin project structure | ||
| Plugin jar can be structured in any way you like. It can be a plain java project, but also a spring/boot based project or based on any other framework. | ||
| It is composed from several parts: | ||
| - class(es) implementing **LowcoderPlugin** interface | ||
| - class(es) implementing **LowcoderEndpoint** interface, containing endpoint handler functions marked with **@EndpointExtension** annotation. These functions must obey following format: | ||
| ```java | ||
| @EndpointExtension(uri = <endpoint uri>, method = <HTTP method>) | ||
| public Mono<ServerResponse> <handler name>(ServerRequest request) | ||
| { | ||
| ... your endpoint logic implementation | ||
| } | ||
| for example: | ||
| @EndpointExtension(uri = "/hello-world", method = Method.GET) | ||
| public Mono<ServerResponse> helloWorld(ServerRequest request) | ||
| { | ||
| return ServerResponse.ok().body(Mono.just(Hello.builder().message("Hello world!").build()), Hello.class); | ||
| } | ||
| ``` | ||
| - TODO: class(es) impelemting **LowcoderDatasource** interface | ||
| ### LowcoderPlugin implementations | ||
| Methods of interest: | ||
| - **pluginId()** - unique plugin ID - if a plugin with such ID is already loaded, subsequent plugins whith this ID will be ignored | ||
| - **description()** - short plugin description | ||
| - **load(ApplicationContext parentContext)** - is called during plugin startup - this is the place where you should completely initialize your plugin. If initialization fails, return false | ||
| - **unload()** - is called during lowcoder API server shutdown - this is the place where you should release all resources | ||
| - **endpoints()** - needs to contain all initialized **PluginEndpoints** you want to expose, for example: | ||
| ```java | ||
| @Override | ||
| public List<PluginEndpoint> endpoints() | ||
| { | ||
| List<PluginEndpoint> endpoints = new ArrayList<>(); | ||
| endpoints.add(new HelloWorldEndpoint()); | ||
| return endpoints; | ||
| } | ||
| ``` | ||
| - **pluginInfo()** - should return a record object with additional information about your plugin. It is serialized to JSON as part of the **/plugins** listing (see **"info"** object in this example): | ||
| ```json | ||
| [ | ||
| { | ||
| "id": "example-plugin", | ||
| "description": "Example plugin for lowcoder platform", | ||
| "info": {} | ||
| }, | ||
| { | ||
| "id": "enterprise", | ||
| "description": "Lowcoder enterprise plugin", | ||
| "info": { | ||
| "enabledFeatures": [ | ||
| "endpointApiUsage" | ||
| ] | ||
| } | ||
| } | ||
| ] | ||
| ``` | ||
| ## TODOs | ||
| 1. Implement endpoint security - currently all plugin endpoints are public (probably by adding **security** attribute to **@EndpointExtension** and enforcing it) | ||
| ## QUESTIONS / CONSIDERATIONS | ||
| 1. currently the plugin endpoints are prefixed with **/plugin/{pluginId}/** - this is hardcoded, do we want to make it configurable? | ||
84 changes: 84 additions & 0 deletionsserver/api-service/distribution/pom.xml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" | ||
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
| <parent> | ||
| <groupId>org.lowcoder</groupId> | ||
| <artifactId>lowcoder-root</artifactId> | ||
| <version>${revision}</version> | ||
| </parent> | ||
| <artifactId>distribution</artifactId> | ||
| <packaging>pom</packaging> | ||
| <properties> | ||
| <assembly.lib.directory>${project.build.directory}/dependencies</assembly.lib.directory> | ||
| </properties> | ||
| <!-- Dependency added here only to make sure this module is built after | ||
| everything alse was built --> | ||
| <dependencies> | ||
| <dependency> | ||
| <groupId>org.lowcoder</groupId> | ||
| <artifactId>lowcoder-sdk</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.lowcoder</groupId> | ||
| <artifactId>lowcoder-infra</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.lowcoder</groupId> | ||
| <artifactId>lowcoder-domain</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.lowcoder</groupId> | ||
| <artifactId>lowcoder-server</artifactId> | ||
| </dependency> | ||
| </dependencies> | ||
| <build> | ||
| <finalName>lowcoder-api-service</finalName> | ||
| <plugins> | ||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> | ||
| <artifactId>maven-dependency-plugin</artifactId> | ||
| <executions> | ||
| <execution> | ||
| <id>copy-dependencies</id> | ||
| <phase>prepare-package</phase> | ||
| <goals> | ||
| <goal>copy-dependencies</goal> | ||
| </goals> | ||
| <configuration> | ||
| <outputDirectory>${assembly.lib.directory}</outputDirectory> | ||
| <overWriteReleases>false</overWriteReleases> | ||
| <overWriteSnapshots>false</overWriteSnapshots> | ||
| <overWriteIfNewer>true</overWriteIfNewer> | ||
| <prependGroupId>true</prependGroupId> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| <plugin> | ||
| <artifactId>maven-assembly-plugin</artifactId> | ||
| <executions> | ||
| <execution> | ||
| <id>distro-assembly</id> | ||
| <phase>package</phase> | ||
| <goals> | ||
| <goal>single</goal> | ||
| </goals> | ||
| <configuration> | ||
| <attach>false</attach> | ||
| <descriptors> | ||
| <descriptor>src/assembly/bin.xml</descriptor> | ||
| </descriptors> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
| </project> |
72 changes: 72 additions & 0 deletionsserver/api-service/distribution/src/assembly/bin.xml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| <assembly xmlns="http://maven.apache.org/ASSEMBLY/2.2.0" | ||
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.2.0 http://maven.apache.org/xsd/assembly-2.2.0.xsd"> | ||
| <id>bin</id> | ||
| <formats> | ||
| <format>dir</format> | ||
| </formats> | ||
| <includeBaseDirectory>false</includeBaseDirectory> | ||
| <files> | ||
| <file> | ||
| <source>src/assembly/set-classpath.sh</source> | ||
| <outputDirectory></outputDirectory> | ||
| </file> | ||
| </files> | ||
| <fileSets> | ||
| <fileSet> | ||
| <directory>${assembly.lib.directory}</directory> | ||
| <outputDirectory>dependencies</outputDirectory> | ||
| <excludes> | ||
| <exclude>${project.groupId}:*</exclude> | ||
| </excludes> | ||
| </fileSet> | ||
| </fileSets> | ||
| <moduleSets> | ||
| <!-- Main lowcoder API server application --> | ||
| <moduleSet> | ||
| <useAllReactorProjects>true</useAllReactorProjects> | ||
| <includes> | ||
| <include>org.lowcoder:lowcoder-server</include> | ||
| </includes> | ||
| <binaries> | ||
| <outputDirectory>app</outputDirectory> | ||
| <includeDependencies>false</includeDependencies> | ||
| <unpack>false</unpack> | ||
| </binaries> | ||
| </moduleSet> | ||
| <!-- Lowcoder API server dependencies --> | ||
| <moduleSet> | ||
| <useAllReactorProjects>true</useAllReactorProjects> | ||
| <includes> | ||
| <include>org.lowcoder:lowcoder-domain</include> | ||
| <include>org.lowcoder:lowcoder-infra</include> | ||
| <include>org.lowcoder:lowcoder-sdk</include> | ||
| </includes> | ||
| <binaries> | ||
| <outputDirectory>libs</outputDirectory> | ||
| <includeDependencies>false</includeDependencies> | ||
| <unpack>false</unpack> | ||
| </binaries> | ||
| </moduleSet> | ||
| <!-- Lowcoder plugins --> | ||
| <moduleSet> | ||
| <useAllReactorProjects>true</useAllReactorProjects> | ||
| <includeSubModules>true</includeSubModules> | ||
| <includes> | ||
| <include>org.lowcoder:*Plugin</include> | ||
| </includes> | ||
| <excludes> | ||
| <exclude>org.lowcoder:sqlBasedPlugin</exclude> | ||
| </excludes> | ||
| <binaries> | ||
| <outputDirectory>plugins</outputDirectory> | ||
| <includeDependencies>false</includeDependencies> | ||
| <unpack>false</unpack> | ||
| </binaries> | ||
| </moduleSet> | ||
| </moduleSets> | ||
| </assembly> |
11 changes: 11 additions & 0 deletionsserver/api-service/distribution/src/assembly/set-classpath.sh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| #!/bin/bash | ||
| # | ||
| # Set lowcoder api service classpath for use in startup script | ||
| # | ||
| export LOWCODER_CLASSPATH="`find libs/ dependencies/ app/ -type f -name "*.jar" | tr '\n' ':' | sed -e 's/:$//'`" | ||
| # | ||
| # Example usage: | ||
| # | ||
| # java -cp "${LOWCODER_CLASSPATH}" org.lowcoder.api.ServerApplication |
Oops, something went wrong.
Uh oh!
There was an error while loading.Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.