feat: add LMStudio, Mise support, fix chat input multiline cursor navigation issue

[sohaieb]

Changes

• Add Mise support
• Add LMStudio nodes support
• Fix an issue when navigating between multilines of chat textarea
• Fix add user input to chat history

Important note

All LMStudio nodes work well, however the LMStudio embedding are unstable: the vector column values are always stored with "0" values.
After some researches and tries, I found out that this issue is due to the LMStudio returned response :

"usage": {
        "prompt_tokens": 0,
        "total_tokens": 0
    }

Not sure but It seems like Followise bases on these values?
I tried to simulate the same embedding process with Ollama but it returns correct values.

Can someone confirm that please ?

BTW I opened an issue to LMStudio basing on this topic, please check it out HERE

[gemini-code-assist]

Summary of Changes

Hello @sohaieb, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the platform's capabilities by integrating LMStudio for local large language model operations, providing users with more flexibility in managing their LLM workflows. Additionally, it incorporates Mise for streamlined development environment setup, ensuring consistent project dependencies across different environments.

Highlights

• LMStudio Integration: Introduced comprehensive support for LMStudio, enabling the use of local large language models for both chat and embedding functionalities within the application. This includes a new credential type for LM Studio API keys and dedicated nodes for chat models, LLMs, and embeddings.
• Mise Support: Added configuration for Mise, a tool version manager, specifying Node.js version 20 to ensure consistent development environment setup.

Changelog

• mise.toml
  • Added configuration for Node.js version 20.
• packages/components/credentials/LmStudioApi.credential.ts
  • Added a new credential class for LM Studio API keys.
• packages/components/nodes/chatmodels/ChatLmStudio/ChatLmStudio.ts
  • Implemented a new chat model node for LM Studio, allowing configuration of various parameters like temperature, model name, and streaming.
• packages/components/nodes/chatmodels/ChatLmStudio/FlowiseChatLmStudio.ts
  • Created a custom wrapper class, FlowiseChatLmStudio, extending Langchain's ChatOpenAI to integrate LM Studio chat models with Flowise-specific functionalities like multi-modal options.
• packages/components/nodes/embeddings/LMStudioAIEmbedding/LmStudioEmbedding.ts
  • Introduced a new embedding node for LM Studio, enabling the generation of local embeddings with configurable base URL and model name.
• packages/components/nodes/llms/LmStudio/LmStudio.ts
  • Added a new LLM node for LM Studio, providing a wrapper for LM Studio large language models with extensive parameter customization.

Activity

• The author noted that LMStudio embeddings are currently unstable, with vector column values consistently showing "0".
• The instability is attributed to LMStudio's API response returning prompt_tokens and total_tokens as 0, which might be critical for how Flowise processes embeddings.
• The author requested confirmation from reviewers regarding the dependency on these token usage values for embedding functionality.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds support for Mise and LMStudio, which is a great addition. The implementation is solid, introducing new credentials and nodes for chat, LLM, and embeddings. I've made a few suggestions to improve the robustness and consistency of the new LMStudio nodes. Specifically, I've pointed out a couple of places where unsafe parsing of numeric inputs could lead to runtime issues and suggested a more robust way to handle optional parameters. I also recommended making the credential for the LMStudio LLM node optional to align with the other LMStudio nodes and common usage patterns. Overall, great work on expanding Flowise's capabilities.

[HenryHengZJ]

what is Mise? why is it needed here?

[sohaieb]

Hi dear @HenryHengZJ ,

what is Mise? why is it needed here?

Mise (Official Docs) helps manage tools like Node and Python per project. It installs and configures them locally and prevents version conflicts.

Please check the docs and let me know if we should include it in Flowise, so I can keep or remove it from the PR.