perf: update pymyssql

feat: add a new piico gm alg

.github/ISSUE_TEMPLATE/----.md

@@ -1,35 +0,0 @@
----
-name: 需求建议
-about: 提出针对本项目的想法和建议
-title: "[Feature] 需求标题"
-labels: 类型:需求
-assignees: 
-  - ibuler
-  - baijiangjie
----
-
-## 注意
-_针对过于简单的需求描述不予考虑。请确保提供足够的细节和信息以支持功能的开发和实现。_
-
-## 功能名称
-[在这里输入功能的名称或标题]
-
-## 功能描述
-[在这里描述该功能的详细内容，包括其作用、目的和所需的功能]
-
-## 用户故事（可选）
-[如果适用，可以提供用户故事来更好地理解该功能的使用场景和用户期望]
-
-## 功能要求
-- [要求1：描述该功能的具体要求，如界面设计、交互逻辑等]
-- [要求2：描述该功能的另一个具体要求]
-- [以此类推，列出所有相关的功能要求]
-
-## 示例或原型（可选）
-[如果有的话，提供该功能的示例或原型图以更好地说明功能的实现方式]
-
-## 优先级
-[描述该功能的优先级，如高、中、低，或使用数字等其他标识]
-
-## 备注（可选）
-[在这里添加任何其他相关信息或备注]

.github/ISSUE_TEMPLATE/1_bug_report.yml

@@ -0,0 +1,72 @@
+name: '🐛 Bug Report'
+description: 'Report an Bug'
+title: '[Bug] '
+labels: ['🐛 Bug']
+assignees: 
+  - baijiangjie
+body:
+  - type: input
+    attributes:
+      label: 'Product Version'
+      description: The versions prior to v2.28 (inclusive) are no longer supported.
+    validations:
+      required: true
+
+  - type: checkboxes
+    attributes:
+      label: 'Product Edition'
+      options:
+        - label: 'Community Edition'
+        - label: 'Enterprise Edition'
+        - label: 'Enterprise Trial Edition'
+    validations:
+      required: true
+    
+  - type: checkboxes
+    attributes:
+      label: 'Installation Method'
+      options:
+        - label: 'Online Installation (One-click command installation)'
+        - label: 'Offline Package Installation'
+        - label: 'All-in-One'
+        - label: '1Panel'
+        - label: 'Kubernetes'
+        - label: 'Source Code'
+
+  - type: textarea
+    attributes:
+      label: 'Environment Information'
+      description: Please provide a clear and concise description outlining your environment information.
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: '🐛 Bug Description'
+      description:
+        Please provide a clear and concise description of the defect. If the issue is complex, please provide detailed explanations. <br/>
+        Unclear descriptions will not be processed. Please ensure you provide enough detail and information to support replicating and fixing the defect.
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: 'Recurrence Steps'
+      description: Please provide a clear and concise description outlining how to reproduce the issue.
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: 'Expected Behavior'
+      description: Please provide a clear and concise description of what you expect to happen.
+
+  - type: textarea
+    attributes:
+      label: 'Additional Information'
+      description: Please add any additional background information about the issue here.
+
+  - type: textarea
+    attributes:
+      label: 'Attempted Solutions'
+      description: If you have already attempted to solve the issue, please list the solutions you have tried here.

.github/ISSUE_TEMPLATE/1_bug_report_cn.yml

@@ -0,0 +1,72 @@
+name: '🐛 反馈缺陷'
+description: '反馈一个缺陷'
+title: '[Bug] '
+labels: ['🐛 Bug']
+assignees: 
+  - baijiangjie
+body:
+  - type: input
+    attributes:
+      label: '产品版本'
+      description: 不再支持 v2.28（含）之前的版本。
+    validations:
+      required: true
+
+  - type: checkboxes
+    attributes:
+      label: '版本类型'
+      options:
+        - label: '社区版'
+        - label: '企业版'
+        - label: '企业试用版'
+    validations:
+      required: true
+    
+  - type: checkboxes
+    attributes:
+      label: '安装方式'
+      options:
+        - label: '在线安装 (一键命令安装)'
+        - label: '离线包安装'
+        - label: 'All-in-One'
+        - label: '1Panel'
+        - label: 'Kubernetes'
+        - label: '源码安装'
+
+  - type: textarea
+    attributes:
+      label: '环境信息'
+      description: 请提供一个清晰且简洁的描述，说明你的环境信息。
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: '🐛 缺陷描述'
+      description: |
+        请提供一个清晰且简洁的缺陷描述，如果问题比较复杂，也请详细说明。<br/> 
+        针对不清晰的描述信息将不予处理。请确保提供足够的细节和信息，以支持对缺陷进行复现和修复。
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: '复现步骤'
+      description: 请提供一个清晰且简洁的描述，说明如何复现问题。
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: '期望结果'
+      description: 请提供一个清晰且简洁的描述，说明你期望发生什么。
+
+  - type: textarea
+    attributes:
+      label: '补充信息'
+      description: 在这里添加关于问题的任何其他背景信息。
+      
+  - type: textarea
+    attributes:
+      label: '尝试过的解决方案'
+      description: 如果你已经尝试解决问题，请在此列出你尝试过的解决方案。

.github/ISSUE_TEMPLATE/2_feature_request.yml

@@ -0,0 +1,56 @@
+name: '⭐️ Feature Request'
+description: 'Suggest an idea'
+title: '[Feature] '
+labels: ['⭐️ Feature Request']
+assignees: 
+  - baijiangjie
+  - ibuler
+body:
+  - type: input
+    attributes:
+      label: 'Product Version'
+      description: The versions prior to v2.28 (inclusive) are no longer supported.
+    validations:
+      required: true
+
+  - type: checkboxes
+    attributes:
+      label: 'Product Edition'
+      options:
+        - label: 'Community Edition'
+        - label: 'Enterprise Edition'
+        - label: 'Enterprise Trial Edition'
+    validations:
+      required: true
+    
+  - type: checkboxes
+    attributes:
+      label: 'Installation Method'
+      options:
+        - label: 'Online Installation (One-click command installation)'
+        - label: 'Offline Package Installation'
+        - label: 'All-in-One'
+        - label: '1Panel'
+        - label: 'Kubernetes'
+        - label: 'Source Code'
+
+  - type: textarea
+    attributes:
+      label: '⭐️ Feature Description'
+      description: |
+        Please add a clear and concise description of the problem you aim to solve with this feature request.<br/>
+        Unclear descriptions will not be processed.      
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: 'Proposed Solution'
+      description: Please provide a clear and concise description of the solution you desire.
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: 'Additional Information'
+      description: Please add any additional background information about the issue here.

.github/ISSUE_TEMPLATE/2_feature_request_cn.yml

@@ -0,0 +1,56 @@
+name: '⭐️ 功能需求'
+description: '提出需求或建议'
+title: '[Feature] '
+labels: ['⭐️ Feature Request']
+assignees: 
+  - baijiangjie
+  - ibuler
+body:
+  - type: input
+    attributes:
+      label: '产品版本'
+      description: 不再支持 v2.28（含）之前的版本。
+    validations:
+      required: true
+
+  - type: checkboxes
+    attributes:
+      label: '版本类型'
+      options:
+        - label: '社区版'
+        - label: '企业版'
+        - label: '企业试用版'
+    validations:
+      required: true
+    
+  - type: checkboxes
+    attributes:
+      label: '安装方式'
+      options:
+        - label: '在线安装 (一键命令安装)'
+        - label: '离线包安装'
+        - label: 'All-in-One'
+        - label: '1Panel'
+        - label: 'Kubernetes'
+        - label: '源码安装'
+
+  - type: textarea
+    attributes:
+      label: '⭐️ 需求描述'
+      description: |
+        请添加一个清晰且简洁的问题描述，阐述你希望通过这个功能需求解决的问题。<br/>
+        针对不清晰的描述信息将不予处理。
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: '解决方案'
+      description: 请清晰且简洁地描述你想要的解决方案。
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: '补充信息'
+      description: 在这里添加关于问题的任何其他背景信息。

.github/ISSUE_TEMPLATE/3_question.yml

@@ -0,0 +1,60 @@
+name: '🤔 Question'
+description: 'Pose a question'
+title: '[Question] '
+labels: ['🤔 Question']
+assignees: 
+  - baijiangjie
+body:
+  - type: input
+    attributes:
+      label: 'Product Version'
+      description: The versions prior to v2.28 (inclusive) are no longer supported.
+    validations:
+      required: true
+
+  - type: checkboxes
+    attributes:
+      label: 'Product Edition'
+      options:
+        - label: 'Community Edition'
+        - label: 'Enterprise Edition'
+        - label: 'Enterprise Trial Edition'
+    validations:
+      required: true
+    
+  - type: checkboxes
+    attributes:
+      label: 'Installation Method'
+      options:
+        - label: 'Online Installation (One-click command installation)'
+        - label: 'Offline Package Installation'
+        - label: 'All-in-One'
+        - label: '1Panel'
+        - label: 'Kubernetes'
+        - label: 'Source Code'
+
+  - type: textarea
+    attributes:
+      label: 'Environment Information'
+      description: Please provide a clear and concise description outlining your environment information.
+    validations:
+      required: true
+      
+  - type: textarea
+    attributes:
+      label: '🤔 Question Description'
+      description: |
+        Please provide a clear and concise description of the defect. If the issue is complex, please provide detailed explanations. <br/>
+        Unclear descriptions will not be processed. 
+    validations:
+      required: true
+      
+  - type: textarea
+    attributes:
+      label: 'Expected Behavior'
+      description: Please provide a clear and concise description of what you expect to happen.
+
+  - type: textarea
+    attributes:
+      label: 'Additional Information'
+      description: Please add any additional background information about the issue here.

.github/ISSUE_TEMPLATE/3_question_cn.yml

@@ -0,0 +1,61 @@
+name: '🤔 问题咨询'
+description: '提出一个问题'
+title: '[Question] '
+labels: ['🤔 Question']
+assignees: 
+  - baijiangjie
+body:
+  - type: input
+    attributes:
+      label: '产品版本'
+      description: 不再支持 v2.28（含）之前的版本。
+    validations:
+      required: true
+
+  - type: checkboxes
+    attributes:
+      label: '版本类型'
+      options:
+        - label: '社区版'
+        - label: '企业版'
+        - label: '企业试用版'
+    validations:
+      required: true
+    
+  - type: checkboxes
+    attributes:
+      label: '安装方式'
+      options:
+        - label: '在线安装 (一键命令安装)'
+        - label: '离线包安装'
+        - label: 'All-in-One'
+        - label: '1Panel'
+        - label: 'Kubernetes'
+        - label: '源码安装'
+
+  - type: textarea
+    attributes:
+      label: '环境信息'
+      description: 请在此详细描述你的环境信息，如操作系统、浏览器和部署架构等。
+    validations:
+      required: true
+
+  - type: textarea
+    attributes:
+      label: '🤔 问题描述'
+      description: |
+        请提供一个清晰且简洁的问题描述，如果问题比较复杂，也请详细说明。<br/> 
+        针对不清晰的描述信息将不予处理。
+    validations:
+      required: true
+      
+  - type: textarea
+    attributes:
+      label: '期望结果'
+      description: 请提供一个清晰且简洁的描述，说明你期望发生什么。
+
+  - type: textarea
+    attributes:
+      label: '补充信息'
+      description: 在这里添加关于问题的任何其他背景信息。
+

.github/ISSUE_TEMPLATE/bug---.md

@@ -1,51 +0,0 @@
----
-name: Bug 提交
-about: 提交产品缺陷帮助我们更好的改进
-title: "[Bug] Bug 标题"
-labels: 类型:Bug
-assignees: 
-   - baijiangjie
----
-
-## 注意
-**JumpServer 版本( v2.28 之前的版本不再支持 )** <br>
-_针对过于简单的 Bug 描述不予考虑。请确保提供足够的细节和信息以支持 Bug 的复现和修复。_
-
-## 当前使用的 JumpServer 版本 (必填)
-[在这里输入当前使用的 JumpServer 的版本号]
-
-## 使用的版本类型 (必填)
-- [ ] 社区版
-- [ ] 企业版
-- [ ] 企业试用版
-
-
-## 版本安装方式 (必填)
-- [ ] 在线安装 (一键命令)
-- [ ] 离线安装 (下载离线包)
-- [ ] All-in-One
-- [ ] 1Panel 安装
-- [ ] Kubernetes 安装
-- [ ] 源码安装
-
-## Bug 描述 (详细)
-[在这里描述 Bug 的详细情况，包括其影响和出现的具体情况]
-
-## 复现步骤
-1. [描述如何复现 Bug 的第一步]
-2. [描述如何复现 Bug 的第二步]
-3. [以此类推，列出所有复现 Bug 所需的步骤]
-
-## 期望行为
-[描述 Bug 出现时期望的系统行为或结果]
-
-## 实际行为
-[描述实际上发生了什么，以及 Bug 出现的具体情况]
-
-## 系统环境
-- 操作系统：[例如：Windows 10, macOS Big Sur]
-- 浏览器/应用版本：[如果适用，请提供相关版本信息]
-- 其他相关环境信息：[如果有其他相关环境信息，请在此处提供]
-
-## 附加信息（可选）
-[在这里添加任何其他相关信息，如截图、错误信息等]

.github/ISSUE_TEMPLATE/question.md

@@ -1,50 +0,0 @@
----
-name: 问题咨询
-about: 提出针对本项目安装部署、使用及其他方面的相关问题
-title: "[Question] 问题标题"
-labels: 类型:提问
-assignees: 
-  - baijiangjie
----
-## 注意
-**请描述您的问题.** <br>
-**JumpServer 版本( v2.28 之前的版本不再支持 )** <br>
-_针对过于简单的 Bug 描述不予考虑。请确保提供足够的细节和信息以支持 Bug 的复现和修复。_
-
-## 当前使用的 JumpServer 版本 (必填)
-[在这里输入当前使用的 JumpServer 的版本号]
-
-## 使用的版本类型 (必填)
-- [ ] 社区版
-- [ ] 企业版
-- [ ] 企业试用版
-
-
-## 版本安装方式 (必填)
-- [ ] 在线安装 (一键命令)
-- [ ] 离线安装 (下载离线包)
-- [ ] All-in-One
-- [ ] 1Panel 安装
-- [ ] Kubernetes 安装
-- [ ] 源码安装
-
-## 问题描述 (详细)
-[在这里描述你遇到的问题]
-
-## 背景信息
-- 操作系统：[例如：Windows 10, macOS Big Sur]
-- 浏览器/应用版本：[如果适用，请提供相关版本信息]
-- 其他相关环境信息：[如果有其他相关环境信息，请在此处提供]
-
-## 具体问题
-[在这里详细描述你的问题，包括任何相关细节或错误信息]
-
-## 尝试过的解决方法
-[如果你已经尝试过解决问题，请在这里列出你已经尝试过的解决方法]
-
-## 预期结果
-[描述你期望的解决方案或结果]
-
-## 我们的期望
-[描述你希望我们提供的帮助或支持]
-

.github/workflows/issue-close-require.yml

@@ -12,7 +12,9 @@ jobs:
         uses: actions-cool/issues-helper@v2
         with:
           actions: 'close-issues'
-          labels: '状态:待反馈'
+          labels: '⏳ Pending feedback'
           inactive-day: 30
           body: |
+            You haven't provided feedback for over 30 days. 
+            We will close this issue. If you have any further needs, you can reopen it or submit a new issue.
             您超过 30 天未反馈信息，我们将关闭该 issue，如有需求您可以重新打开或者提交新的 issue。

.github/workflows/issue-close.yml

@@ -13,4 +13,4 @@ jobs:
         if: ${{ !github.event.issue.pull_request }}
         with:
           actions: 'remove-labels'
-          labels: '状态:待处理,状态:待反馈'
+          labels: '🔔 Pending processing,⏳ Pending feedback'

.github/workflows/issue-comment.yml

@@ -13,13 +13,13 @@ jobs:
         uses: actions-cool/issues-helper@v2
         with:
           actions: 'add-labels'
-          labels: '状态:待处理'
+          labels: '🔔 Pending processing'
 
       - name: Remove require reply label
         uses: actions-cool/issues-helper@v2
         with:
           actions: 'remove-labels'
-          labels: '状态:待反馈'
+          labels: '⏳ Pending feedback'
 
   add-label-if-is-member:
     runs-on: ubuntu-latest
@@ -55,11 +55,11 @@ jobs:
         uses: actions-cool/issues-helper@v2
         with:
           actions: 'add-labels'
-          labels: '状态:待反馈'
+          labels: '⏳ Pending feedback'
 
       - name: Remove require handle label
         if: contains(steps.member_names.outputs.data, github.event.comment.user.login)
         uses: actions-cool/issues-helper@v2
         with:
           actions: 'remove-labels'
-          labels: '状态:待处理'
+          labels: '🔔 Pending processing'

.github/workflows/issue-open.yml

@@ -13,4 +13,4 @@ jobs:
         if: ${{ !github.event.issue.pull_request }}
         with:
           actions: 'add-labels'
-          labels: '状态:待处理'
+          labels: '🔔 Pending processing'

.github/workflows/jms-generic-action-handler.yml

@@ -10,3 +10,4 @@ jobs:
       - uses: jumpserver/action-generic-handler@master
         env:
           GITHUB_TOKEN: ${{ secrets.PRIVATE_TOKEN }}
+          I18N_TOKEN: ${{ secrets.I18N_TOKEN }}

Dockerfile

@@ -0,0 +1,137 @@
+FROM python:3.11-slim-bullseye AS stage-1
+ARG TARGETARCH
+
+ARG VERSION
+ENV VERSION=$VERSION
+
+WORKDIR /opt/jumpserver
+ADD . .
+RUN echo > /opt/jumpserver/config.yml \
+    && cd utils && bash -ixeu build.sh
+
+FROM python:3.11-slim-bullseye as stage-2
+ARG TARGETARCH
+
+ARG BUILD_DEPENDENCIES="              \
+        g++                           \
+        make                          \
+        pkg-config"
+
+ARG DEPENDENCIES="                    \
+        freetds-dev                   \
+        libffi-dev                    \
+        libjpeg-dev                   \
+        libkrb5-dev                   \
+        libldap2-dev                  \
+        libpq-dev                     \
+        libsasl2-dev                  \
+        libssl-dev                    \
+        libxml2-dev                   \
+        libxmlsec1-dev                \
+        libxmlsec1-openssl            \
+        freerdp2-dev                  \
+        libaio-dev"
+
+ARG TOOLS="                           \
+        ca-certificates               \
+        curl                          \
+        default-libmysqlclient-dev    \
+        default-mysql-client          \
+        git                           \
+        git-lfs                       \
+        unzip                         \
+        xz-utils                      \
+        wget"
+
+ARG APT_MIRROR=http://mirrors.ustc.edu.cn
+RUN --mount=type=cache,target=/var/cache/apt,sharing=locked,id=core-apt \
+    --mount=type=cache,target=/var/lib/apt,sharing=locked,id=core-apt \
+    sed -i "s@http://.*.debian.org@${APT_MIRROR}@g" /etc/apt/sources.list \
+    && rm -f /etc/apt/apt.conf.d/docker-clean \
+    && ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
+    && apt-get update \
+    && apt-get -y install --no-install-recommends ${BUILD_DEPENDENCIES} \
+    && apt-get -y install --no-install-recommends ${DEPENDENCIES} \
+    && apt-get -y install --no-install-recommends ${TOOLS} \
+    && echo "no" | dpkg-reconfigure dash
+
+WORKDIR /opt/jumpserver
+
+ARG PIP_MIRROR=https://pypi.tuna.tsinghua.edu.cn/simple
+RUN --mount=type=cache,target=/root/.cache \
+    --mount=type=bind,source=poetry.lock,target=/opt/jumpserver/poetry.lock \
+    --mount=type=bind,source=pyproject.toml,target=/opt/jumpserver/pyproject.toml \
+    set -ex \
+    && python3 -m venv /opt/py3 \
+    && pip install poetry -i ${PIP_MIRROR} \
+    && poetry config virtualenvs.create false \
+    && . /opt/py3/bin/activate \
+    && poetry install
+
+FROM python:3.11-slim-bullseye
+ARG TARGETARCH
+ENV LANG=zh_CN.UTF-8 \
+    PATH=/opt/py3/bin:$PATH
+
+ARG DEPENDENCIES="                    \
+        libjpeg-dev                   \
+        libpq-dev                     \
+        libx11-dev                    \
+        freerdp2-dev                  \
+        libxmlsec1-openssl"
+
+ARG TOOLS="                           \
+        ca-certificates               \
+        curl                          \
+        default-libmysqlclient-dev    \
+        default-mysql-client          \
+        iputils-ping                  \
+        locales                       \
+        netcat-openbsd                \
+        nmap                          \
+        openssh-client                \
+        patch                         \
+        sshpass                       \
+        telnet                        \
+        vim                           \
+        bubblewrap                    \
+        wget"
+
+ARG APT_MIRROR=http://mirrors.ustc.edu.cn
+RUN --mount=type=cache,target=/var/cache/apt,sharing=locked,id=core-apt \
+    --mount=type=cache,target=/var/lib/apt,sharing=locked,id=core-apt \
+    sed -i "s@http://.*.debian.org@${APT_MIRROR}@g" /etc/apt/sources.list \
+    && rm -f /etc/apt/apt.conf.d/docker-clean \
+    && ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
+    && apt-get update \
+    && apt-get -y install --no-install-recommends ${DEPENDENCIES} \
+    && apt-get -y install --no-install-recommends ${TOOLS} \
+    && mkdir -p /root/.ssh/ \
+    && echo "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null\n\tCiphers +aes128-cbc\n\tKexAlgorithms +diffie-hellman-group1-sha1\n\tHostKeyAlgorithms +ssh-rsa" > /root/.ssh/config \
+    && echo "no" | dpkg-reconfigure dash \
+    && echo "zh_CN.UTF-8" | dpkg-reconfigure locales \
+    && sed -i "s@# export @export @g" ~/.bashrc \
+    && sed -i "s@# alias @alias @g" ~/.bashrc
+
+ARG RECEPTOR_VERSION=v1.4.5
+RUN set -ex \
+    && wget -O /opt/receptor.tar.gz https://github.com/ansible/receptor/releases/download/${RECEPTOR_VERSION}/receptor_${RECEPTOR_VERSION/v/}_linux_${TARGETARCH}.tar.gz \
+    && tar -xf /opt/receptor.tar.gz -C /usr/local/bin/ \
+    && chown root:root /usr/local/bin/receptor \
+    && chmod 755 /usr/local/bin/receptor \
+    && rm -f /opt/receptor.tar.gz
+
+COPY --from=stage-2 /opt/py3 /opt/py3
+COPY --from=stage-1 /opt/jumpserver/release/jumpserver /opt/jumpserver
+COPY --from=stage-1 /opt/jumpserver/release/jumpserver/apps/libs/ansible/ansible.cfg /etc/ansible/
+
+WORKDIR /opt/jumpserver
+
+ARG VERSION
+ENV VERSION=$VERSION
+
+VOLUME /opt/jumpserver/data
+
+EXPOSE 8080
+
+ENTRYPOINT ["./entrypoint.sh"]

Dockerfile-ce

@@ -1,4 +1,4 @@
-FROM python:3.11-slim-bullseye as stage-1
+FROM python:3.11-slim-bullseye AS stage-1
 ARG TARGETARCH
 
 ARG VERSION
@@ -94,6 +94,7 @@ ARG TOOLS="                           \
         sshpass                       \
         telnet                        \
         vim                           \
+        bubblewrap                    \
         wget"
 
 ARG APT_MIRROR=http://mirrors.ustc.edu.cn

Dockerfile-ee

@@ -1,5 +1,5 @@
 ARG VERSION
-FROM registry.fit2cloud.com/jumpserver/xpack:${VERSION} as build-xpack
+FROM registry.fit2cloud.com/jumpserver/xpack:${VERSION} AS build-xpack
 FROM registry.fit2cloud.com/jumpserver/core-ce:${VERSION}
 
-COPY --from=build-xpack /opt/xpack /opt/jumpserver/apps/xpack
+COPY --from=build-xpack /opt/xpack /opt/jumpserver/apps/xpack

apps/accounts/automations/change_secret/custom/ssh/main.yml

@@ -13,11 +13,11 @@
         login_password: "{{ jms_account.secret }}"
         login_secret_type: "{{ jms_account.secret_type }}"
         login_private_key_path: "{{ jms_account.private_key_path }}"
-        become: "{{ custom_become | default(False) }}"
-        become_method: "{{ custom_become_method | default('su') }}"
-        become_user: "{{ custom_become_user | default('') }}"
-        become_password: "{{ custom_become_password | default('') }}"
-        become_private_key_path: "{{ custom_become_private_key_path | default(None) }}"
+        become: "{{ jms_custom_become | default(False) }}"
+        become_method: "{{ jms_custom_become_method | default('su') }}"
+        become_user: "{{ jms_custom_become_user | default('') }}"
+        become_password: "{{ jms_custom_become_password | default('') }}"
+        become_private_key_path: "{{ jms_custom_become_private_key_path | default(None) }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default(None) }}"
       register: ping_info
@@ -31,11 +31,11 @@
         login_port: "{{ jms_asset.port }}"
         login_secret_type: "{{ jms_account.secret_type }}"
         login_private_key_path: "{{ jms_account.private_key_path }}"
-        become: "{{ custom_become | default(False) }}"
-        become_method: "{{ custom_become_method | default('su') }}"
-        become_user: "{{ custom_become_user | default('') }}"
-        become_password: "{{ custom_become_password | default('') }}"
-        become_private_key_path: "{{ custom_become_private_key_path | default(None) }}"
+        become: "{{ jms_custom_become | default(False) }}"
+        become_method: "{{ jms_custom_become_method | default('su') }}"
+        become_user: "{{ jms_custom_become_user | default('') }}"
+        become_password: "{{ jms_custom_become_password | default('') }}"
+        become_private_key_path: "{{ jms_custom_become_private_key_path | default(None) }}"
         name: "{{ account.username }}"
         password: "{{ account.secret }}"
         commands: "{{ params.commands }}"

apps/accounts/automations/change_secret/host/aix/main.yml

@@ -35,12 +35,24 @@
         - user_info.failed
         - params.groups
 
+    - name: "Set {{ account.username }} sudo setting"
+      ansible.builtin.lineinfile:
+        dest: /etc/sudoers
+        state: present
+        regexp: "^{{ account.username }} ALL="
+        line: "{{ account.username + ' ALL=(ALL) NOPASSWD: ' + params.sudo }}"
+        validate: visudo -cf %s
+      when:
+        - user_info.failed or params.modify_sudo
+        - params.sudo
+
     - name: "Change {{ account.username }} password"
       ansible.builtin.user:
         name: "{{ account.username }}"
         password: "{{ account.secret | password_hash('des') }}"
         update_password: always
       ignore_errors: true
+      register: change_secret_result
       when: account.secret_type == "password"
 
     - name: remove jumpserver ssh key
@@ -57,19 +69,9 @@
         user: "{{ account.username }}"
         key: "{{ account.secret }}"
         exclusive: "{{ ssh_params.exclusive }}"
+      register: change_secret_result
       when: account.secret_type == "ssh_key"
 
-    - name: "Set {{ account.username }} sudo setting"
-      ansible.builtin.lineinfile:
-        dest: /etc/sudoers
-        state: present
-        regexp: "^{{ account.username }} ALL="
-        line: "{{ account.username + ' ALL=(ALL) NOPASSWD: ' + params.sudo }}"
-        validate: visudo -cf %s
-      when:
-        - user_info.failed
-        - params.sudo
-
     - name: Refresh connection
       ansible.builtin.meta: reset_connection
 
@@ -86,7 +88,9 @@
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
-      when: account.secret_type == "password"
+      when:
+       - account.secret_type == "password"
+       - check_conn_after_change or change_secret_result.failed | default(false)
       delegate_to: localhost
 
     - name: "Verify {{ account.username }} SSH KEY (paramiko)"
@@ -97,5 +101,7 @@
         login_private_key_path: "{{ account.private_key_path  }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
-      when: account.secret_type == "ssh_key"
+      when:
+       - account.secret_type == "ssh_key"
+       - check_conn_after_change or change_secret_result.failed | default(false)
       delegate_to: localhost

apps/accounts/automations/change_secret/host/aix/manifest.yml

@@ -5,6 +5,12 @@ type:
   - AIX
 method: change_secret
 params:
+  - name: modify_sudo
+    type: bool
+    label: "{{ 'Modify sudo label' | trans }}"
+    default: False
+    help_text: "{{ 'Modify params sudo help text' | trans }}"
+
   - name: sudo
     type: str
     label: 'Sudo'
@@ -34,6 +40,11 @@ i18n:
     ja: 'Ansible user モジュールを使用してアカウントのパスワード変更 (DES)'
     en: 'Using Ansible module user to change account secret (DES)'
 
+  Modify params sudo help text:
+    zh: '如果用户存在，可以修改sudo权限'
+    ja: 'ユーザーが存在する場合、sudo権限を変更できます'
+    en: 'If the user exists, sudo permissions can be modified'
+
   Params sudo help text:
     zh: '使用逗号分隔多个命令，如: /bin/whoami,/sbin/ifconfig'
     ja: 'コンマで区切って複数のコマンドを入力してください。例: /bin/whoami,/sbin/ifconfig'
@@ -49,6 +60,11 @@ i18n:
     ja: 'グループを入力してください。複数のグループはコンマで区切ってください（既存のグループを入力してください）'
     en: 'Please enter the group. Multiple groups are separated by commas (please enter the existing group)'
 
+  Modify sudo label:
+    zh: '修改 sudo 权限'
+    ja: 'sudo 権限を変更'
+    en: 'Modify sudo'
+
   Params home label:
     zh: '家目录'
     ja: 'ホームディレクトリ'

apps/accounts/automations/change_secret/host/posix/main.yml

@@ -35,12 +35,24 @@
         - user_info.failed
         - params.groups
 
+    - name: "Set {{ account.username }} sudo setting"
+      ansible.builtin.lineinfile:
+        dest: /etc/sudoers
+        state: present
+        regexp: "^{{ account.username }} ALL="
+        line: "{{ account.username + ' ALL=(ALL) NOPASSWD: ' + params.sudo }}"
+        validate: visudo -cf %s
+      when:
+        - user_info.failed or params.modify_sudo
+        - params.sudo
+
     - name: "Change {{ account.username }} password"
       ansible.builtin.user:
         name: "{{ account.username }}"
         password: "{{ account.secret | password_hash('sha512') }}"
         update_password: always
       ignore_errors: true
+      register: change_secret_result
       when: account.secret_type == "password"
 
     - name: remove jumpserver ssh key
@@ -57,19 +69,9 @@
         user: "{{ account.username }}"
         key: "{{ account.secret }}"
         exclusive: "{{ ssh_params.exclusive }}"
+      register: change_secret_result
       when: account.secret_type == "ssh_key"
 
-    - name: "Set {{ account.username }} sudo setting"
-      ansible.builtin.lineinfile:
-        dest: /etc/sudoers
-        state: present
-        regexp: "^{{ account.username }} ALL="
-        line: "{{ account.username + ' ALL=(ALL) NOPASSWD: ' + params.sudo }}"
-        validate: visudo -cf %s
-      when:
-        - user_info.failed
-        - params.sudo
-
     - name: Refresh connection
       ansible.builtin.meta: reset_connection
 
@@ -86,7 +88,9 @@
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
-      when: account.secret_type == "password"
+      when:
+       - account.secret_type == "password"
+       - check_conn_after_change or change_secret_result.failed | default(false)
       delegate_to: localhost
 
     - name: "Verify {{ account.username }} SSH KEY (paramiko)"
@@ -97,5 +101,7 @@
         login_private_key_path: "{{ account.private_key_path  }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
-      when: account.secret_type == "ssh_key"
+      when:
+       - account.secret_type == "ssh_key"
+       - check_conn_after_change or change_secret_result.failed | default(false)
       delegate_to: localhost

apps/accounts/automations/change_secret/host/posix/manifest.yml

@@ -6,6 +6,12 @@ type:
   - linux
 method: change_secret
 params:
+  - name: modify_sudo
+    type: bool
+    label: "{{ 'Modify sudo label' | trans }}"
+    default: False
+    help_text: "{{ 'Modify params sudo help text' | trans }}"
+
   - name: sudo
     type: str
     label: 'Sudo'
@@ -36,6 +42,11 @@ i18n:
     ja: 'Ansible user モジュールを使用して アカウントのパスワード変更 (SHA512)'
     en: 'Using Ansible module user to change account secret (SHA512)'
 
+  Modify params sudo help text:
+    zh: '如果用户存在，可以修改sudo权限'
+    ja: 'ユーザーが存在する場合、sudo権限を変更できます'
+    en: 'If the user exists, sudo permissions can be modified'
+
   Params sudo help text:
     zh: '使用逗号分隔多个命令，如: /bin/whoami,/sbin/ifconfig'
     ja: 'コンマで区切って複数のコマンドを入力してください。例: /bin/whoami,/sbin/ifconfig'
@@ -51,6 +62,11 @@ i18n:
     ja: 'グループを入力してください。複数のグループはコンマで区切ってください（既存のグループを入力してください）'
     en: 'Please enter the group. Multiple groups are separated by commas (please enter the existing group)'
 
+  Modify sudo label:
+    zh: '修改 sudo 权限'
+    ja: 'sudo 権限を変更'
+    en: 'Modify sudo'
+
   Params home label:
     zh: '家目录'
     ja: 'ホームディレクトリ'

apps/accounts/automations/change_secret/manager.py

@@ -134,6 +134,7 @@ class ChangeSecretManager(AccountBasePlaybookManager):
                 record_id = self.record_map[asset_account_id]
                 try:
                     recorder = ChangeSecretRecord.objects.get(id=record_id)
+                    new_secret = recorder.new_secret
                 except ChangeSecretRecord.DoesNotExist:
                     print(f"Record {record_id} not found")
                     continue

apps/accounts/automations/push_account/host/aix/main.yml

@@ -35,12 +35,24 @@
         - user_info.failed
         - params.groups
 
+    - name: "Set {{ account.username }} sudo setting"
+      ansible.builtin.lineinfile:
+        dest: /etc/sudoers
+        state: present
+        regexp: "^{{ account.username }} ALL="
+        line: "{{ account.username + ' ALL=(ALL) NOPASSWD: ' + params.sudo }}"
+        validate: visudo -cf %s
+      when:
+        - user_info.failed or params.modify_sudo
+        - params.sudo
+
     - name: "Change {{ account.username }} password"
       ansible.builtin.user:
         name: "{{ account.username }}"
         password: "{{ account.secret | password_hash('des') }}"
         update_password: always
       ignore_errors: true
+      register: change_secret_result
       when: account.secret_type == "password"
 
     - name: remove jumpserver ssh key
@@ -57,19 +69,9 @@
         user: "{{ account.username }}"
         key: "{{ account.secret }}"
         exclusive: "{{ ssh_params.exclusive }}"
+      register: change_secret_result
       when: account.secret_type == "ssh_key"
 
-    - name: "Set {{ account.username }} sudo setting"
-      ansible.builtin.lineinfile:
-        dest: /etc/sudoers
-        state: present
-        regexp: "^{{ account.username }} ALL="
-        line: "{{ account.username + ' ALL=(ALL) NOPASSWD: ' + params.sudo }}"
-        validate: visudo -cf %s
-      when:
-        - user_info.failed
-        - params.sudo
-
     - name: Refresh connection
       ansible.builtin.meta: reset_connection
 
@@ -86,7 +88,9 @@
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
-      when: account.secret_type == "password"
+      when:
+       - account.secret_type == "password"
+       - check_conn_after_change or change_secret_result.failed | default(false)
       delegate_to: localhost
 
     - name: "Verify {{ account.username }} SSH KEY (paramiko)"
@@ -97,6 +101,8 @@
         login_private_key_path: "{{ account.private_key_path  }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
-      when: account.secret_type == "ssh_key"
+      when:
+       - account.secret_type == "ssh_key"
+       - check_conn_after_change or change_secret_result.failed | default(false)
       delegate_to: localhost
 

apps/accounts/automations/push_account/host/aix/manifest.yml

@@ -5,6 +5,12 @@ type:
   - AIX
 method: push_account
 params:
+  - name: modify_sudo
+    type: bool
+    label: "{{ 'Modify sudo label' | trans }}"
+    default: False
+    help_text: "{{ 'Modify params sudo help text' | trans }}"
+
   - name: sudo
     type: str
     label: 'Sudo'
@@ -34,6 +40,11 @@ i18n:
     ja: 'Ansible user モジュールを使用して Aix アカウントをプッシュする (DES)'
     en: 'Using Ansible module user to push account (DES)'
 
+  Modify params sudo help text:
+    zh: '如果用户存在，可以修改sudo权限'
+    ja: 'ユーザーが存在する場合、sudo権限を変更できます'
+    en: 'If the user exists, sudo permissions can be modified'
+
   Params sudo help text:
     zh: '使用逗号分隔多个命令，如: /bin/whoami,/sbin/ifconfig'
     ja: 'コンマで区切って複数のコマンドを入力してください。例: /bin/whoami,/sbin/ifconfig'
@@ -49,6 +60,11 @@ i18n:
     ja: 'グループを入力してください。複数のグループはコンマで区切ってください（既存のグループを入力してください）'
     en: 'Please enter the group. Multiple groups are separated by commas (please enter the existing group)'
 
+  Modify sudo label:
+    zh: '修改 sudo 权限'
+    ja: 'sudo 権限を変更'
+    en: 'Modify sudo'
+
   Params home label:
     zh: '家目录'
     ja: 'ホームディレクトリ'

apps/accounts/automations/push_account/host/posix/main.yml

@@ -35,12 +35,24 @@
         - user_info.failed
         - params.groups
 
+    - name: "Set {{ account.username }} sudo setting"
+      ansible.builtin.lineinfile:
+        dest: /etc/sudoers
+        state: present
+        regexp: "^{{ account.username }} ALL="
+        line: "{{ account.username + ' ALL=(ALL) NOPASSWD: ' + params.sudo }}"
+        validate: visudo -cf %s
+      when:
+        - user_info.failed or params.modify_sudo
+        - params.sudo
+
     - name: "Change {{ account.username }} password"
       ansible.builtin.user:
         name: "{{ account.username }}"
         password: "{{ account.secret | password_hash('sha512') }}"
         update_password: always
       ignore_errors: true
+      register: change_secret_result
       when: account.secret_type == "password"
 
     - name: remove jumpserver ssh key
@@ -57,19 +69,9 @@
         user: "{{ account.username }}"
         key: "{{ account.secret }}"
         exclusive: "{{ ssh_params.exclusive }}"
+      register: change_secret_result
       when: account.secret_type == "ssh_key"
 
-    - name: "Set {{ account.username }} sudo setting"
-      ansible.builtin.lineinfile:
-        dest: /etc/sudoers
-        state: present
-        regexp: "^{{ account.username }} ALL="
-        line: "{{ account.username + ' ALL=(ALL) NOPASSWD: ' + params.sudo }}"
-        validate: visudo -cf %s
-      when:
-        - user_info.failed
-        - params.sudo
-
     - name: Refresh connection
       ansible.builtin.meta: reset_connection
 
@@ -86,7 +88,9 @@
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
-      when: account.secret_type == "password"
+      when:
+       - account.secret_type == "password"
+       - check_conn_after_change or change_secret_result.failed | default(false)
       delegate_to: localhost
 
     - name: "Verify {{ account.username }} SSH KEY (paramiko)"
@@ -97,6 +101,8 @@
         login_private_key_path: "{{ account.private_key_path  }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
-      when: account.secret_type == "ssh_key"
+      when:
+       - account.secret_type == "ssh_key"
+       - check_conn_after_change or change_secret_result.failed | default(false)
       delegate_to: localhost
 

apps/accounts/automations/push_account/host/posix/manifest.yml

@@ -6,6 +6,12 @@ type:
   - linux
 method: push_account
 params:
+  - name: modify_sudo
+    type: bool
+    label: "{{ 'Modify sudo label' | trans }}"
+    default: False
+    help_text: "{{ 'Modify params sudo help text' | trans }}"
+
   - name: sudo
     type: str
     label: 'Sudo'
@@ -36,6 +42,11 @@ i18n:
     ja: 'Ansible user モジュールを使用してアカウントをプッシュする (sha512)'
     en: 'Using Ansible module user to push account (sha512)'
 
+  Modify params sudo help text:
+    zh: '如果用户存在，可以修改sudo权限'
+    ja: 'ユーザーが存在する場合、sudo権限を変更できます'
+    en: 'If the user exists, sudo permissions can be modified'
+
   Params sudo help text:
     zh: '使用逗号分隔多个命令，如: /bin/whoami,/sbin/ifconfig'
     ja: 'コンマで区切って複数のコマンドを入力してください。例: /bin/whoami,/sbin/ifconfig'
@@ -51,6 +62,11 @@ i18n:
     ja: 'グループを入力してください。複数のグループはコンマで区切ってください（既存のグループを入力してください）'
     en: 'Please enter the group. Multiple groups are separated by commas (please enter the existing group)'
 
+  Modify sudo label:
+    zh: '修改 sudo 权限'
+    ja: 'sudo 権限を変更'
+    en: 'Modify sudo'
+
   Params home label:
     zh: '家目录'
     ja: 'ホームディレクトリ'

apps/accounts/automations/remove_account/host/posix/main.yml

@@ -12,11 +12,13 @@
         path: "{{ user_home_dir.stdout }}"
       register: home_dir
       when: user_home_dir.stdout != ""
+      ignore_errors: yes
 
     - name: "Rename user home directory if it exists"
       ansible.builtin.command:
         cmd: "mv {{ user_home_dir.stdout }} {{ user_home_dir.stdout }}.bak"
       when: home_dir.stat | default(false) and user_home_dir.stdout != ""
+      ignore_errors: yes
 
     - name: "Remove account"
       ansible.builtin.user:

apps/accounts/automations/remove_account/host/windows/main.yml

@@ -4,6 +4,4 @@
     - name: "Remove account"
       ansible.windows.win_user:
         name: "{{ account.username }}"
-        state: absent
-        purge: yes
-        force: yes
+        state: absent

apps/accounts/automations/verify_account/host/posix/main.yml

@@ -8,6 +8,7 @@
         ansible_user: "{{ account.username }}"
         ansible_password: "{{ account.secret }}"
         ansible_ssh_private_key_file: "{{ account.private_key_path }}"
+        ansible_timeout: 30
       when: not account.become.ansible_become
 
     - name: Verify account connectivity(Switch)
@@ -20,4 +21,5 @@
         ansible_become_method: "{{ account.become.ansible_become_method }}"
         ansible_become_user: "{{ account.become.ansible_become_user }}"
         ansible_become_password: "{{ account.become.ansible_become_password }}"
+        ansible_timeout: 30
       when: account.become.ansible_become

apps/accounts/automations/verify_account/host/windows/main.yml

@@ -9,3 +9,4 @@
       vars:
         ansible_user: "{{ account.username }}"
         ansible_password: "{{ account.secret }}"
+        ansible_timeout: 30

apps/accounts/models/account.py

@@ -53,7 +53,8 @@ class Account(AbsConnectivity, LabeledMixin, BaseAccount):
         on_delete=models.SET_NULL, verbose_name=_("Su from")
     )
     version = models.IntegerField(default=0, verbose_name=_('Version'))
-    history = AccountHistoricalRecords(included_fields=['id', '_secret', 'secret_type', 'version'])
+    history = AccountHistoricalRecords(included_fields=['id', '_secret', 'secret_type', 'version'],
+                                       verbose_name=_("historical Account"))
     source = models.CharField(max_length=30, default=Source.LOCAL, verbose_name=_('Source'))
     source_id = models.CharField(max_length=128, null=True, blank=True, verbose_name=_('Source ID'))
 
@@ -119,7 +120,8 @@ class Account(AbsConnectivity, LabeledMixin, BaseAccount):
             return auth
 
         auth.update(self.make_account_ansible_vars(su_from))
-        become_method = platform.su_method if platform.su_method else 'sudo'
+
+        become_method = platform.ansible_become_method
         password = su_from.secret if become_method == 'sudo' else self.secret
         auth['ansible_become'] = True
         auth['ansible_become_method'] = become_method

apps/accounts/serializers/account/account.py

@@ -79,18 +79,28 @@ class AccountCreateUpdateSerializerMixin(serializers.Serializer):
 
     @staticmethod
     def get_template_attr_for_account(template):
-        # Set initial data from template
         field_names = [
-            'name', 'username', 'secret',
-            'secret_type', 'privileged', 'is_active'
+            'name', 'username',
+            'secret_type', 'secret',
+            'privileged', 'is_active'
         ]
 
+        field_map = {
+            'push_params': 'params',
+            'auto_push': 'push_now'
+        }
+
+        field_names.extend(field_map.keys())
+
         attrs = {}
         for name in field_names:
             value = getattr(template, name, None)
             if value is None:
                 continue
-            attrs[name] = value
+
+            attr_name = field_map.get(name, name)
+            attrs[attr_name] = value
+
         attrs['secret'] = template.get_secret()
         return attrs
 
@@ -173,7 +183,8 @@ class AccountCreateUpdateSerializerMixin(serializers.Serializer):
         params = validated_data.pop('params', None)
         self.clean_auth_fields(validated_data)
         instance, stat = self.do_create(validated_data)
-        self.push_account_if_need(instance, push_now, params, stat)
+        if instance.source == Source.LOCAL:
+            self.push_account_if_need(instance, push_now, params, stat)
         return instance
 
     def update(self, instance, validated_data):
@@ -275,8 +286,8 @@ class AssetAccountBulkSerializer(
         fields = [
             'name', 'username', 'secret', 'secret_type', 'passphrase',
             'privileged', 'is_active', 'comment', 'template',
-            'on_invalid', 'push_now', 'assets', 'su_from_username',
-            'source', 'source_id',
+            'on_invalid', 'push_now', 'params', 'assets',
+            'su_from_username', 'source', 'source_id',
         ]
         extra_kwargs = {
             'name': {'required': False},
@@ -414,16 +425,23 @@ class AssetAccountBulkSerializer(
         return results
 
     @staticmethod
-    def push_accounts_if_need(results, push_now):
+    def push_accounts_if_need(results, push_now, params):
         if not push_now:
             return
-        accounts = [str(v['instance']) for v in results if v.get('instance')]
-        push_accounts_to_assets_task.delay(accounts)
+
+        account_ids = [v['instance'] for v in results if v.get('instance')]
+        accounts = Account.objects.filter(id__in=account_ids, source=Source.LOCAL)
+        if not accounts.exists():
+            return
+
+        account_ids = [str(_id) for _id in accounts.values_list('id', flat=True)]
+        push_accounts_to_assets_task.delay(account_ids, params)
 
     def create(self, validated_data):
+        params = validated_data.pop('params', None)
         push_now = validated_data.pop('push_now', False)
         results = self.perform_bulk_create(validated_data)
-        self.push_accounts_if_need(results, push_now)
+        self.push_accounts_if_need(results, push_now, params)
         for res in results:
             res['asset'] = str(res['asset'])
         return results

apps/accounts/signal_handlers.py

@@ -6,6 +6,7 @@ from django.dispatch import receiver
 from django.utils.translation import gettext_noop
 
 from accounts.backends import vault_client
+from accounts.const import Source
 from audits.const import ActivityChoices
 from audits.signal_handlers import create_activities
 from common.decorators import merge_delay_run
@@ -32,7 +33,7 @@ def push_accounts_if_need(accounts=()):
     template_accounts = defaultdict(list)
     for ac in accounts:
         # 再强调一次吧
-        if ac.source != 'template':
+        if ac.source != Source.TEMPLATE:
             continue
         template_accounts[ac.source_id].append(ac)
 
@@ -61,7 +62,7 @@ def create_accounts_activities(account, action='create'):
 
 @receiver(post_save, sender=Account)
 def on_account_create_by_template(sender, instance, created=False, **kwargs):
-    if not created or instance.source != 'template':
+    if not created:
         return
     push_accounts_if_need.delay(accounts=(instance,))
     create_accounts_activities(instance, action='create')

apps/assets/api/asset/asset.py

@@ -292,6 +292,7 @@ class AssetsTaskCreateApi(AssetsTaskMixin, generics.CreateAPIView):
     def check_permissions(self, request):
         action_perm_require = {
             "refresh": "assets.refresh_assethardwareinfo",
+            "test": "assets.test_assetconnectivity",
         }
         _action = request.data.get("action")
         perm_required = action_perm_require.get(_action)

apps/assets/api/tree.py

@@ -39,16 +39,16 @@ class NodeChildrenApi(generics.ListCreateAPIView):
         self.instance = self.get_object()
 
     def perform_create(self, serializer):
+        data = serializer.validated_data
+        _id = data.get("id")
+        value = data.get("value")
+        if value:
+            children = self.instance.get_children()
+            if children.filter(value=value).exists():
+                raise JMSException(_('The same level node name cannot be the same'))
+        else:
+            value = self.instance.get_next_child_preset_name()
         with NodeAddChildrenLock(self.instance):
-            data = serializer.validated_data
-            _id = data.get("id")
-            value = data.get("value")
-            if value:
-                children = self.instance.get_children()
-                if children.filter(value=value).exists():
-                    raise JMSException(_('The same level node name cannot be the same'))
-            else:
-                value = self.instance.get_next_child_preset_name()
             node = self.instance.create_child(value=value, _id=_id)
             # 避免查询 full value
             node._full_value = node.value
@@ -126,7 +126,7 @@ class NodeChildrenAsTreeApi(SerializeToTreeNodeMixin, NodeChildrenApi):
         include_assets = self.request.query_params.get('assets', '0') == '1'
         if not self.instance or not include_assets:
             return Asset.objects.none()
-        if self.instance.is_org_root():
+        if not self.request.GET.get('search') and self.instance.is_org_root():
             return Asset.objects.none()
         if query_all:
             assets = self.instance.get_all_assets()

apps/assets/automations/base/manager.py

@@ -11,6 +11,7 @@ from django.utils.translation import gettext as _
 from sshtunnel import SSHTunnelForwarder
 
 from assets.automations.methods import platform_automation_methods
+from common.db.utils import safe_db_connection
 from common.utils import get_logger, lazyproperty, is_openssh_format_key, ssh_pubkey_gen
 from ops.ansible import JMSInventory, DefaultCallback, SuperPlaybookRunner
 from ops.ansible.interface import interface
@@ -37,7 +38,7 @@ class SSHTunnelManager:
         info = self.file_to_json(runner.inventory)
         servers, not_valid = [], []
         for k, host in info['all']['hosts'].items():
-            jms_asset, jms_gateway = host.get('jms_asset'), host.get('gateway')
+            jms_asset, jms_gateway = host.get('jms_asset'), host.get('jms_gateway')
             if not jms_gateway:
                 continue
             try:
@@ -113,11 +114,7 @@ class BasePlaybookManager:
         if not data:
             data = automation_params.get(method_id, {})
         params = serializer(data).data
-        return {
-            field_name: automation_params.get(field_name, '')
-            if not params[field_name] else params[field_name]
-            for field_name in params
-        }
+        return params
 
     @property
     def platform_automation_methods(self):
@@ -191,6 +188,7 @@ class BasePlaybookManager:
             host['error'] = _('{} disabled'.format(self.__class__.method_type()))
             return host
 
+        host['check_conn_after_change'] = settings.CHECK_CONN_AFTER_CHANGE
         host = self.convert_cert_to_file(host, kwargs.get('path_dir'))
         host['params'] = self.get_params(automation, method_type)
         return host
@@ -342,7 +340,8 @@ class BasePlaybookManager:
             try:
                 kwargs.update({"clean_workspace": False})
                 cb = runner.run(**kwargs)
-                self.on_runner_success(runner, cb)
+                with safe_db_connection():
+                    self.on_runner_success(runner, cb)
             except Exception as e:
                 self.on_runner_failed(runner, e)
             finally:

apps/assets/automations/ping/custom/ssh/main.yml

@@ -14,11 +14,11 @@
         login_port: "{{ jms_asset.port }}"
         login_secret_type: "{{ jms_account.secret_type }}"
         login_private_key_path: "{{ jms_account.private_key_path }}"
-        become: "{{ custom_become | default(False) }}"
-        become_method: "{{ custom_become_method | default('su') }}"
-        become_user: "{{ custom_become_user | default('') }}"
-        become_password: "{{ custom_become_password | default('') }}"
-        become_private_key_path: "{{ custom_become_private_key_path | default(None) }}"
+        become: "{{ jms_custom_become | default(False) }}"
+        become_method: "{{ jms_custom_become_method | default('su') }}"
+        become_user: "{{ jms_custom_become_user | default('') }}"
+        become_password: "{{ jms_custom_become_password | default('') }}"
+        become_private_key_path: "{{ jms_custom_become_private_key_path | default(None) }}"
         old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default(None) }}"
 

apps/assets/automations/ping/host/posix/main.yml

@@ -1,5 +1,7 @@
 - hosts: demo
   gather_facts: no
+  vars:
+    ansible_timeout: 30
   tasks:
     - name: Posix ping
       ansible.builtin.ping:

apps/assets/automations/ping/host/windows/main.yml

@@ -1,5 +1,7 @@
 - hosts: windows
   gather_facts: no
+  vars:
+    ansible_timeout: 30
   tasks:
     - name: Refresh connection
       ansible.builtin.meta: reset_connection

apps/assets/const/__init__.py

@@ -2,5 +2,6 @@ from .automation import *
 from .base import *
 from .category import *
 from .host import *
+from .platform import *
 from .protocol import *
 from .types import *

apps/assets/const/database.py

@@ -8,6 +8,7 @@ class DatabaseTypes(BaseType):
     ORACLE = 'oracle', 'Oracle'
     SQLSERVER = 'sqlserver', 'SQLServer'
     DB2 = 'db2', 'DB2'
+    DAMENG = 'dameng', 'Dameng'
     CLICKHOUSE = 'clickhouse', 'ClickHouse'
     MONGODB = 'mongodb', 'MongoDB'
     REDIS = 'redis', 'Redis'
@@ -55,6 +56,15 @@ class DatabaseTypes(BaseType):
                 'change_secret_enabled': False,
                 'push_account_enabled': False,
             },
+            cls.DAMENG: {
+                'ansible_enabled': False,
+                'ping_enabled': False,
+                'gather_facts_enabled': False,
+                'gather_accounts_enabled': False,
+                'verify_account_enabled': False,
+                'change_secret_enabled': False,
+                'push_account_enabled': False,
+            },
             cls.CLICKHOUSE: {
                 'ansible_enabled': False,
                 'ping_enabled': False,
@@ -84,6 +94,7 @@ class DatabaseTypes(BaseType):
             cls.ORACLE: [{'name': 'Oracle'}],
             cls.SQLSERVER: [{'name': 'SQLServer'}],
             cls.DB2: [{'name': 'DB2'}],
+            cls.DAMENG: [{'name': 'Dameng'}],
             cls.CLICKHOUSE: [{'name': 'ClickHouse'}],
             cls.MONGODB: [{'name': 'MongoDB'}],
             cls.REDIS: [

apps/assets/const/host.py

@@ -19,7 +19,7 @@ class HostTypes(BaseType):
                 'charset': 'utf-8',  # default
                 'domain_enabled': True,
                 'su_enabled': True,
-                'su_methods': ['sudo', 'su'],
+                'su_methods': ['sudo', 'su', 'only_sudo', 'only_su'],
             },
             cls.WINDOWS: {
                 'su_enabled': False,

apps/assets/const/platform.py

@@ -0,0 +1,11 @@
+from django.db.models import TextChoices
+
+
+class SuMethodChoices(TextChoices):
+    sudo = "sudo", "sudo su -"
+    su = "su", "su - "
+    only_sudo = "only_sudo", "sudo su"
+    only_su = "only_su", "su"
+    enable = "enable", "enable"
+    super = "super", "super 15"
+    super_level = "super_level", "super level 15"

apps/assets/const/protocol.py

@@ -23,6 +23,7 @@ class Protocol(ChoicesMixin, models.TextChoices):
     postgresql = 'postgresql', 'PostgreSQL'
     sqlserver = 'sqlserver', 'SQLServer'
     db2 = 'db2', 'DB2'
+    dameng = 'dameng', 'Dameng'
     clickhouse = 'clickhouse', 'ClickHouse'
     redis = 'redis', 'Redis'
     mongodb = 'mongodb', 'MongoDB'
@@ -185,6 +186,12 @@ class Protocol(ChoicesMixin, models.TextChoices):
                 'secret_types': ['password'],
                 'xpack': True,
             },
+            cls.dameng: {
+                'port': 5236,
+                'required': True,
+                'secret_types': ['password'],
+                'xpack': True,
+            },
             cls.clickhouse: {
                 'port': 9000,
                 'required': True,
@@ -201,6 +208,12 @@ class Protocol(ChoicesMixin, models.TextChoices):
                         'default': 'admin',
                         'label': _('Auth source'),
                         'help_text': _('The database to authenticate against')
+                    },
+                    'connection_options': {
+                        'type': 'str',
+                        'default': '',
+                        'label': _('Connection options'),
+                        'help_text': _('The connection specific options eg. retryWrites=false&retryReads=false')
                     }
                 }
             },
@@ -282,23 +295,17 @@ class Protocol(ChoicesMixin, models.TextChoices):
                 'setting': {
                     'api_mode': {
                         'type': 'choice',
-                        'default': 'gpt-3.5-turbo',
+                        'default': 'gpt-4o-mini',
                         'label': _('API mode'),
                         'choices': [
-                            ('gpt-3.5-turbo', 'GPT-3.5 Turbo'),
-                            ('gpt-3.5-turbo-1106', 'GPT-3.5 Turbo 1106'),
+                            ('gpt-4o-mini', 'GPT-4o-mini'),
+                            ('gpt-4o', 'GPT-4o'),
+                            ('gpt-4-turbo', 'GPT-4 Turbo'),
                         ]
                     }
                 }
             }
         }
-        if settings.XPACK_LICENSE_IS_VALID:
-            choices = protocols[cls.chatgpt]['setting']['api_mode']['choices']
-            choices.extend([
-                ('gpt-4', 'GPT-4'),
-                ('gpt-4-turbo', 'GPT-4 Turbo'),
-                ('gpt-4o', 'GPT-4o'),
-            ])
         return protocols
 
     @classmethod

apps/assets/migrations/0107_automation.py

@@ -1,10 +1,12 @@
 # Generated by Django 3.2.16 on 2022-12-30 08:08
 
-import common.db.fields
-from django.db import migrations, models
-import django.db.models.deletion
 import uuid
 
+import django.db.models.deletion
+from django.db import migrations, models
+
+import common.db.fields
+
 
 class Migration(migrations.Migration):
 
@@ -53,7 +55,7 @@ class Migration(migrations.Migration):
             ],
             options={
                 'verbose_name': 'Automation task execution',
-                'ordering': ('-date_start',),
+                'ordering': ('org_id', '-date_start',),
             },
         ),
         migrations.CreateModel(

apps/assets/migrations/0128_auto_20240514_1521.py

@@ -0,0 +1,31 @@
+# Generated by Django 4.1.10 on 2023-10-07 06:37
+
+from django.db import migrations
+
+
+def add_dameng_platform(apps, schema_editor):
+    platform_cls = apps.get_model('assets', 'Platform')
+    automation_cls = apps.get_model('assets', 'PlatformAutomation')
+    platform, _ = platform_cls.objects.update_or_create(
+        name='Dameng', defaults={
+            'name': 'Dameng', 'category': 'database',
+            'internal': True, 'type': 'dameng',
+            'domain_enabled': True, 'su_enabled': False,
+            'su_method': None, 'comment': 'Dameng', 'created_by': 'System',
+            'updated_by': 'System', 'custom_fields': []
+        }
+    )
+    platform.protocols.update_or_create(name='dameng', defaults={
+        'name': 'dameng', 'port': 5236, 'primary': True, 'setting': {}
+    })
+    automation_cls.objects.update_or_create(platform=platform, defaults={'ansible_enabled': False})
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ('assets', '0127_automation_remove_account'),
+    ]
+
+    operations = [
+        migrations.RunPython(add_dameng_platform)
+    ]

apps/assets/models/asset/common.py

@@ -174,7 +174,7 @@ class Asset(NodesRelationMixin, LabeledMixin, AbsConnectivity, JSONFilterMixin,
 
     def get_labels(self):
         from labels.models import Label, LabeledResource
-        res_type = ContentType.objects.get_for_model(self.__class__)
+        res_type = ContentType.objects.get_for_model(self.__class__.label_model())
         label_ids = LabeledResource.objects.filter(res_type=res_type, res_id=self.id) \
             .values_list('label_id', flat=True)
         return Label.objects.filter(id__in=label_ids)

apps/assets/models/automations/base.py

@@ -123,7 +123,7 @@ class AutomationExecution(OrgModelMixin):
     )
 
     class Meta:
-        ordering = ('-date_start',)
+        ordering = ('org_id', '-date_start',)
         verbose_name = _('Automation task execution')
 
     @property

apps/assets/models/platform.py

@@ -1,7 +1,7 @@
 from django.db import models
 from django.utils.translation import gettext_lazy as _
 
-from assets.const import AllTypes, Category, Protocol
+from assets.const import AllTypes, Category, Protocol, SuMethodChoices
 from common.db.fields import JsonDictTextField
 from common.db.models import JMSBaseModel
 
@@ -127,6 +127,17 @@ class Platform(LabeledMixin, JMSBaseModel):
             return True
         return False
 
+    @property
+    def ansible_become_method(self):
+        su_method = self.su_method or SuMethodChoices.sudo
+        if su_method in [SuMethodChoices.sudo, SuMethodChoices.only_sudo]:
+            method = SuMethodChoices.sudo
+        elif su_method in [SuMethodChoices.su, SuMethodChoices.only_su]:
+            method = SuMethodChoices.su
+        else:
+            method = su_method
+        return method
+
     def __str__(self):
         return self.name
 

apps/assets/serializers/asset/common.py

@@ -323,7 +323,9 @@ class AssetSerializer(BulkOrgResourceModelSerializer, ResourceLabelsMixin, Writa
             template_id = data.get('template', None)
             if template_id:
                 template = AccountTemplate.objects.get(id=template_id)
-                if template and template.su_from:
+                template.push_params = data.pop('push_params', {})
+                data['params'] = template.push_params
+                if template.su_from:
                     su_from_name_username_secret_type_map[template.name] = (
                         template.su_from.username, template.su_from.secret_type
                     )
@@ -381,6 +383,7 @@ class AssetSerializer(BulkOrgResourceModelSerializer, ResourceLabelsMixin, Writa
 
 
 class DetailMixin(serializers.Serializer):
+    accounts = AssetAccountSerializer(many=True, required=False, label=_('Accounts'))
     spec_info = MethodSerializer(label=_('Spec info'), read_only=True)
     gathered_info = MethodSerializer(label=_('Gathered info'), read_only=True)
     auto_config = serializers.DictField(read_only=True, label=_('Auto info'))
@@ -395,7 +398,7 @@ class DetailMixin(serializers.Serializer):
     def get_field_names(self, declared_fields, info):
         names = super().get_field_names(declared_fields, info)
         names.extend([
-            'gathered_info', 'spec_info', 'auto_config',
+            'accounts', 'gathered_info', 'spec_info', 'auto_config',
         ])
         return names
 

apps/assets/serializers/asset/database.py

@@ -41,7 +41,7 @@ class DatabaseSerializer(AssetSerializer):
         elif self.context.get('request'):
             platform_id = self.context['request'].query_params.get('platform')
 
-        if not platform and platform_id:
+        if not platform and platform_id and str(platform_id).isdigit():
             platform = Platform.objects.filter(id=platform_id).first()
         return platform
 

apps/assets/serializers/platform.py

@@ -9,7 +9,7 @@ from common.serializers import (
 )
 from common.serializers.fields import LabeledChoiceField
 from common.utils import lazyproperty
-from ..const import Category, AllTypes, Protocol
+from ..const import Category, AllTypes, Protocol, SuMethodChoices
 from ..models import Platform, PlatformProtocol, PlatformAutomation
 
 __all__ = ["PlatformSerializer", "PlatformOpsMethodSerializer", "PlatformProtocolSerializer"]
@@ -124,13 +124,6 @@ class PlatformCustomField(serializers.Serializer):
 
 
 class PlatformSerializer(ResourceLabelsMixin, WritableNestedModelSerializer):
-    SU_METHOD_CHOICES = [
-        ("sudo", "sudo su -"),
-        ("su", "su - "),
-        ("enable", "enable"),
-        ("super", "super 15"),
-        ("super_level", "super level 15")
-    ]
     id = serializers.IntegerField(
         label='ID', required=False,
         validators=[UniqueValidator(queryset=Platform.objects.all())]
@@ -141,8 +134,8 @@ class PlatformSerializer(ResourceLabelsMixin, WritableNestedModelSerializer):
     protocols = PlatformProtocolSerializer(label=_("Protocols"), many=True, required=False)
     automation = PlatformAutomationSerializer(label=_("Automation"), required=False, default=dict)
     su_method = LabeledChoiceField(
-        choices=SU_METHOD_CHOICES, label=_("Su method"),
-        required=False, default="sudo", allow_null=True
+        choices=SuMethodChoices.choices, label=_("Su method"),
+        required=False, default=SuMethodChoices.sudo, allow_null=True
     )
     custom_fields = PlatformCustomField(label=_("Custom fields"), many=True, required=False)
 

apps/audits/api.py

@@ -28,7 +28,7 @@ from orgs.utils import current_org, tmp_to_root_org
 from rbac.permissions import RBACPermission
 from terminal.models import default_storage
 from users.models import User
-from .backends import TYPE_ENGINE_MAPPING
+from .backends import get_operate_log_storage
 from .const import ActivityChoices
 from .filters import UserSessionFilterSet, OperateLogFilterSet
 from .models import (
@@ -146,7 +146,9 @@ class MyLoginLogViewSet(UserLoginCommonMixin, OrgReadonlyModelViewSet):
 
     def get_queryset(self):
         qs = super().get_queryset()
-        qs = qs.filter(username=self.request.user.username)
+        username = self.request.user.username
+        q = Q(username=username) | Q(username__icontains=f'({username})')
+        qs = qs.filter(q)
         return qs
 
 
@@ -187,9 +189,13 @@ class ResourceActivityAPIView(generics.ListAPIView):
             'id', 'datetime', 'r_detail', 'r_detail_id',
             'r_user', 'r_action', 'r_type'
         )
-        org_q = Q(org_id=Organization.SYSTEM_ID) | Q(org_id=current_org.id)
-        if resource_id:
-            org_q |= Q(org_id='') | Q(org_id=Organization.ROOT_ID)
+
+        org_q = Q()
+        if not current_org.is_root():
+            org_q = Q(org_id=Organization.SYSTEM_ID) | Q(org_id=current_org.id)
+            if resource_id:
+                org_q |= Q(org_id='') | Q(org_id=Organization.ROOT_ID)
+
         with tmp_to_root_org():
             qs1 = self.get_operate_log_qs(fields, limit, org_q, resource_id=resource_id)
             qs2 = self.get_activity_log_qs(fields, limit, org_q, resource_id=resource_id)
@@ -222,13 +228,11 @@ class OperateLogViewSet(OrgReadonlyModelViewSet):
         if self.is_action_detail:
             with tmp_to_root_org():
                 qs |= OperateLog.objects.filter(org_id=Organization.SYSTEM_ID)
-        es_config = settings.OPERATE_LOG_ELASTICSEARCH_CONFIG
-        if es_config:
-            engine_mod = import_module(TYPE_ENGINE_MAPPING['es'])
-            store = engine_mod.OperateLogStore(es_config)
-            if store.ping(timeout=2):
-                qs = ESQuerySet(store)
-                qs.model = OperateLog
+
+        storage = get_operate_log_storage()
+        if storage.get_type() == 'es':
+            qs = ESQuerySet(storage)
+            qs.model = OperateLog
         return qs
 
 

apps/audits/backends/__init__.py

@@ -1,18 +1,62 @@
-from importlib import import_module
-
 from django.conf import settings
+from django.core.cache import cache
+from django.utils.translation import gettext_lazy as _
+
+from common.utils import get_logger
+from .base import BaseOperateStorage
+from .es import OperateLogStore as ESOperateLogStore
+from .db import OperateLogStore as DBOperateLogStore
 
 
-TYPE_ENGINE_MAPPING = {
-    'db': 'audits.backends.db',
-    'es': 'audits.backends.es',
+logger = get_logger(__file__)
+
+_global_op_log_storage: None | ESOperateLogStore | DBOperateLogStore = None
+op_log_type_mapping = {
+    'server': DBOperateLogStore, 'es': ESOperateLogStore
 }
 
 
-def get_operate_log_storage(default=False):
-    engine_mod = import_module(TYPE_ENGINE_MAPPING['db'])
-    es_config = settings.OPERATE_LOG_ELASTICSEARCH_CONFIG
-    if not default and es_config:
-        engine_mod = import_module(TYPE_ENGINE_MAPPING['es'])
-    storage = engine_mod.OperateLogStore(es_config)
-    return storage
+def _send_es_unavailable_alarm_msg():
+    from terminal.notifications import StorageConnectivityMessage
+    from terminal.const import CommandStorageType
+
+    key = 'OPERATE_LOG_ES_UNAVAILABLE_KEY'
+    if cache.get(key):
+        return
+
+    cache.set(key, 1, 60)
+    errors = [{
+        'msg': _("Connect failed"), 'name': f"{_('Operate log')}",
+        'type': CommandStorageType.es.label
+    }]
+    StorageConnectivityMessage(errors).publish_async()
+
+
+def refresh_log_storage():
+    global _global_op_log_storage
+    _global_op_log_storage = None
+
+    if settings.OPERATE_LOG_ELASTICSEARCH_CONFIG.get('HOSTS'):
+        try:
+            config = settings.OPERATE_LOG_ELASTICSEARCH_CONFIG
+            log_storage = op_log_type_mapping['es'](config)
+            _global_op_log_storage = log_storage
+        except Exception as e:
+            _send_es_unavailable_alarm_msg()
+            logger.warning('Invalid logs storage type: es, error: %s' % str(e))
+
+    if not _global_op_log_storage:
+        _global_op_log_storage = op_log_type_mapping['server']()
+
+
+def get_operate_log_storage():
+    if _global_op_log_storage is None:
+        refresh_log_storage()
+
+    log_storage = _global_op_log_storage
+    if not log_storage.ping(timeout=3):
+        if log_storage.get_type() == 'es':
+            _send_es_unavailable_alarm_msg()
+        logger.warning('Switch default operate log storage.')
+        log_storage = op_log_type_mapping['server']()
+    return log_storage

apps/audits/backends/base.py

@@ -0,0 +1,15 @@
+from perms.const import ActionChoices
+
+
+class BaseOperateStorage(object):
+    @staticmethod
+    def get_type():
+        return 'base'
+
+    @staticmethod
+    def _get_special_handler(resource_type):
+        # 根据资源类型，处理特殊字段
+        resource_map = {
+            'Asset permission': lambda k, v: ActionChoices.display(int(v)) if k == 'Actions' else v
+        }
+        return resource_map.get(resource_type, lambda k, v: v)

apps/audits/backends/db.py

@@ -2,14 +2,14 @@
 from django.utils.translation import gettext_lazy as _
 
 from audits.models import OperateLog
-from perms.const import ActionChoices
+from .base import BaseOperateStorage
 
 
-class OperateLogStore(object):
+class OperateLogStore(BaseOperateStorage):
     # 用不可见字符分割前后数据，节省存储-> diff: {'key': 'before\0after'}
     SEP = '\0'
 
-    def __init__(self, config):
+    def __init__(self, *args, **kwargs):
         self.model = OperateLog
         self.max_length = 2048
         self.max_length_tip_msg = _(
@@ -17,9 +17,13 @@ class OperateLogStore(object):
         )
 
     @staticmethod
-    def ping(timeout=None):
+    def ping(*args, **kwargs):
         return True
 
+    @staticmethod
+    def get_type():
+        return 'db'
+
     @classmethod
     def convert_before_after_to_diff(cls, before, after):
         if not isinstance(before, dict):
@@ -46,18 +50,13 @@ class OperateLogStore(object):
             before[k], after[k] = before_value, after_value
         return before, after
 
-    @staticmethod
-    def _get_special_handler(resource_type):
-        # 根据资源类型，处理特殊字段
-        resource_map = {
-            'Asset permission': lambda k, v: ActionChoices.display(int(v)) if k == 'Actions' else v
-        }
-        return resource_map.get(resource_type, lambda k, v: v)
-
     @classmethod
     def convert_diff_friendly(cls, op_log):
         diff_list = list()
         handler = cls._get_special_handler(op_log.resource_type)
+        # 标记翻译字符串
+        labels = _("labels")
+        operate_log_id = _("operate_log_id")
         for k, v in op_log.diff.items():
             before, after = v.split(cls.SEP, 1)
             diff_list.append({

apps/audits/backends/es.py

@@ -2,16 +2,17 @@
 #
 import uuid
 
+from django.utils.translation import gettext_lazy as _
+
 from common.utils.timezone import local_now_display
 from common.utils import get_logger
-from common.utils.encode import Singleton
 from common.plugins.es import ES
-
+from .base import BaseOperateStorage
 
 logger = get_logger(__file__)
 
 
-class OperateLogStore(ES, metaclass=Singleton):
+class OperateLogStore(BaseOperateStorage, ES):
     def __init__(self, config):
         properties = {
             "id": {
@@ -48,7 +49,26 @@ class OperateLogStore(ES, metaclass=Singleton):
         self.pre_use_check()
 
     @staticmethod
-    def make_data(data):
+    def get_type():
+        return 'es'
+
+    @classmethod
+    def convert_diff_friendly(cls, op_log):
+        diff_list = []
+        handler = cls._get_special_handler(op_log.get('resource_type'))
+        before = op_log.get('before') or {}
+        after = op_log.get('after') or {}
+        keys = set(before.keys()) | set(after.keys())
+        for key in keys:
+            before_v, after_v = before.get(key), after.get(key)
+            diff_list.append({
+                'field': _(key),
+                'before': handler(key, before_v) if before_v else _('empty'),
+                'after': handler(key, after_v) if after_v else _('empty'),
+            })
+        return diff_list
+
+    def make_data(self, data):
         op_id = data.get('id', str(uuid.uuid4()))
         datetime_param = data.get('datetime', local_now_display())
         data = {

apps/audits/const.py

@@ -37,6 +37,9 @@ class ActionChoices(TextChoices):
     approve = 'approve', _('Approve')
     close = 'close', _('Close')
 
+    # Custom action
+    finished = 'finished', _('Finished')
+
 
 class LoginTypeChoices(TextChoices):
     web = "W", _("Web")

apps/audits/handler.py

@@ -7,7 +7,6 @@ from django.utils.translation import gettext_lazy as _
 
 from common.local import encrypted_field_set
 from common.utils import get_request_ip, get_logger
-from common.utils.encode import Singleton
 from common.utils.timezone import as_current_tz
 from jumpserver.utils import current_request
 from orgs.models import Organization
@@ -21,17 +20,9 @@ from .backends import get_operate_log_storage
 logger = get_logger(__name__)
 
 
-class OperatorLogHandler(metaclass=Singleton):
+class OperatorLogHandler(object):
     CACHE_KEY = 'OPERATOR_LOG_CACHE_KEY'
 
-    def __init__(self):
-        self.log_client = self.get_storage_client()
-
-    @staticmethod
-    def get_storage_client():
-        client = get_operate_log_storage()
-        return client
-
     @staticmethod
     def _consistent_type_to_str(value1, value2):
         if isinstance(value1, datetime):
@@ -58,7 +49,7 @@ class OperatorLogHandler(metaclass=Singleton):
             return
 
         key = '%s_%s' % (self.CACHE_KEY, instance_id)
-        cache.set(key, instance_dict, 3 * 60)
+        cache.set(key, instance_dict, 3)
 
     def get_instance_dict_from_cache(self, instance_id):
         if instance_id is None:
@@ -164,13 +155,8 @@ class OperatorLogHandler(metaclass=Singleton):
             'remote_addr': remote_addr, 'before': before, 'after': after,
         }
         with transaction.atomic():
-            if self.log_client.ping(timeout=1):
-                client = self.log_client
-            else:
-                logger.info('Switch default operate log storage save.')
-                client = get_operate_log_storage(default=True)
-
             try:
+                client = get_operate_log_storage()
                 client.save(**data)
             except Exception as e:
                 error_msg = 'An error occurred saving OperateLog.' \

apps/audits/migrations/0023_auto_20230906_1322.py

@@ -19,7 +19,7 @@ class Migration(migrations.Migration):
         migrations.AlterField(
             model_name='operatelog',
             name='action',
-            field=models.CharField(choices=[('view', 'View'), ('update', 'Update'), ('delete', 'Delete'), ('create', 'Create'), ('download', 'Download'), ('connect', 'Connect'), ('login', 'Login'), ('change_password', 'Change password'), ('accept', 'Accept'), ('review', 'Review'), ('notice', 'Notifications'), ('reject', 'Reject'), ('approve', 'Approve'), ('close', 'Close')], max_length=16, verbose_name='Action'),
+            field=models.CharField(choices=[('view', 'View'), ('update', 'Update'), ('delete', 'Delete'), ('create', 'Create'), ('download', 'Download'), ('connect', 'Connect'), ('login', 'Login'), ('change_password', 'Change password'), ('accept', 'Accept'), ('review', 'Review'), ('notice', 'Notifications'), ('reject', 'Reject'), ('approve', 'Approve'), ('close', 'Close'), ('finished', 'Finished')], max_length=16, verbose_name='Action'),
         ),
         migrations.AlterField(
             model_name='userloginlog',

apps/audits/models.py

@@ -257,6 +257,8 @@ class UserLoginLog(models.Model):
 
 
 class UserSession(models.Model):
+    _OPERATE_LOG_ACTION = {'delete': ActionChoices.finished}
+
     id = models.UUIDField(default=uuid.uuid4, primary_key=True)
     ip = models.GenericIPAddressField(verbose_name=_("Login IP"))
     key = models.CharField(max_length=128, verbose_name=_("Session key"))

apps/audits/serializers.py

@@ -3,7 +3,7 @@
 from django.utils.translation import gettext_lazy as _
 from rest_framework import serializers
 
-from audits.backends.db import OperateLogStore
+from audits.backends import get_operate_log_storage
 from common.serializers.fields import LabeledChoiceField, ObjectRelatedField
 from common.utils import reverse, i18n_trans
 from common.utils.timezone import as_current_tz
@@ -77,7 +77,7 @@ class OperateLogActionDetailSerializer(serializers.ModelSerializer):
         fields = ('diff',)
 
     def to_representation(self, instance):
-        return {'diff': OperateLogStore.convert_diff_friendly(instance)}
+        return {'diff': get_operate_log_storage().convert_diff_friendly(instance)}
 
 
 class OperateLogSerializer(BulkOrgResourceModelSerializer):

apps/audits/signal_handlers/operate_log.py

@@ -3,7 +3,9 @@
 import uuid
 
 from django.apps import apps
-from django.db.models.signals import post_save, pre_save, m2m_changed, pre_delete
+from django.db.models.signals import (
+    pre_delete, pre_save, m2m_changed, post_delete, post_save
+)
 from django.dispatch import receiver
 from django.utils import translation
 
@@ -12,7 +14,7 @@ from audits.handler import (
     create_or_update_operate_log, get_instance_dict_from_cache
 )
 from audits.utils import model_to_dict_for_operate_log as model_to_dict
-from common.const.signals import POST_ADD, POST_REMOVE, POST_CLEAR, SKIP_SIGNAL
+from common.const.signals import POST_ADD, POST_REMOVE, POST_CLEAR, OP_LOG_SKIP_SIGNAL
 from common.signals import django_ready
 from jumpserver.utils import current_request
 from ..const import MODELS_NEED_RECORD, ActionChoices
@@ -75,7 +77,7 @@ def signal_of_operate_log_whether_continue(
     condition = True
     if not instance:
         condition = False
-    if instance and getattr(instance, SKIP_SIGNAL, False):
+    if instance and getattr(instance, OP_LOG_SKIP_SIGNAL, False):
         condition = False
     # 不记录组件的操作日志
     user = current_request.user if current_request else None
@@ -94,7 +96,7 @@ def signal_of_operate_log_whether_continue(
     return condition
 
 
-@receiver(pre_save)
+@receiver([pre_save, pre_delete])
 def on_object_pre_create_or_update(
         sender, instance=None, raw=False, using=None, update_fields=None, **kwargs
 ):
@@ -103,6 +105,7 @@ def on_object_pre_create_or_update(
     )
     if not ok:
         return
+
     with translation.override('en'):
         # users.PrivateToken Model 没有 id 有 pk字段
         instance_id = getattr(instance, 'id', getattr(instance, 'pk', None))
@@ -145,7 +148,7 @@ def on_object_created_or_update(
         )
 
 
-@receiver(pre_delete)
+@receiver(post_delete)
 def on_object_delete(sender, instance=None, **kwargs):
     ok = signal_of_operate_log_whether_continue(sender, instance, False)
     if not ok:
@@ -153,9 +156,15 @@ def on_object_delete(sender, instance=None, **kwargs):
 
     with translation.override('en'):
         resource_type = sender._meta.verbose_name
+        action = getattr(sender, '_OPERATE_LOG_ACTION', {})
+        action = action.get('delete', ActionChoices.delete)
+        instance_id = getattr(instance, 'id', getattr(instance, 'pk', None))
+        log_id, before = get_instance_dict_from_cache(instance_id)
+        if not log_id:
+            log_id, before = None, model_to_dict(instance)
         create_or_update_operate_log(
-            ActionChoices.delete, resource_type,
-            resource=instance, before=model_to_dict(instance)
+            action, resource_type, log_id=log_id,
+            resource=instance, before=before,
         )
 
 
@@ -166,7 +175,7 @@ def on_django_start_set_operate_log_monitor_models(sender, **kwargs):
         'django_celery_beat', 'contenttypes', 'sessions', 'auth',
     }
     exclude_models = {
-        'UserPasswordHistory', 'ContentType',
+        'UserPasswordHistory', 'ContentType', 'Asset',
         'MessageContent', 'SiteMessage',
         'PlatformAutomation', 'PlatformProtocol', 'Protocol',
         'HistoricalAccount', 'GatheredUser', 'ApprovalRule',
@@ -178,13 +187,15 @@ def on_django_start_set_operate_log_monitor_models(sender, **kwargs):
         'PermedAsset', 'PermedAccount', 'MenuPermission',
         'Permission', 'TicketSession', 'ApplyLoginTicket',
         'ApplyCommandTicket', 'ApplyLoginAssetTicket',
-        'FavoriteAsset',
+        'FavoriteAsset', 'ChangeSecretRecord'
     }
+    include_models = set()
     for i, app in enumerate(apps.get_models(), 1):
         app_name = app._meta.app_label
         model_name = app._meta.object_name
         if app_name in exclude_apps or \
                 model_name in exclude_models or \
                 model_name.endswith('Execution'):
-            continue
+            if model_name not in include_models:
+                continue
         MODELS_NEED_RECORD.add(model_name)

apps/audits/tasks.py

@@ -16,6 +16,7 @@ from common.storage.ftp_file import FTPFileStorageHandler
 from common.utils import get_log_keep_day, get_logger
 from ops.celery.decorator import register_as_period_task
 from ops.models import CeleryTaskExecution
+from orgs.utils import tmp_to_root_org
 from terminal.backends import server_replay_storage
 from terminal.models import Session, Command
 from .models import UserLoginLog, OperateLog, FTPLog, ActivityLog, PasswordChangeLog
@@ -105,8 +106,9 @@ def clean_expired_session_period():
     logger.info("Clean session item done")
     batch_delete(expired_commands)
     logger.info("Clean session command done")
-    command = "find %s -mtime +%s \\( -name '*.json' -o -name '*.tar' -o -name '*.gz' \\) -exec rm -f {} \\;" % (
-        replay_dir, days
+    file_types = "-name '*.json' -o -name '*.tar' -o -name '*.gz' -o -name '*.mp4'"
+    command = "find %s -mtime +%s \\( %s \\) -exec rm -f {} \\;" % (
+        replay_dir, days, file_types
     )
     subprocess.call(command, shell=True)
     command = "find %s -type d -empty -delete;" % replay_dir
@@ -118,13 +120,14 @@ def clean_expired_session_period():
 @register_as_period_task(crontab=CRONTAB_AT_AM_TWO)
 def clean_audits_log_period():
     print("Start clean audit session task log")
-    clean_login_log_period()
-    clean_operation_log_period()
-    clean_ftp_log_period()
-    clean_activity_log_period()
-    clean_celery_tasks_period()
-    clean_expired_session_period()
-    clean_password_change_log_period()
+    with tmp_to_root_org():
+        clean_login_log_period()
+        clean_operation_log_period()
+        clean_ftp_log_period()
+        clean_activity_log_period()
+        clean_celery_tasks_period()
+        clean_expired_session_period()
+        clean_password_change_log_period()
 
 
 @shared_task(verbose_name=_('Upload FTP file to external storage'))

apps/audits/utils.py

@@ -49,9 +49,15 @@ def _get_instance_field_value(
                 continue
 
             value = getattr(instance, f.name, None) or getattr(instance, f.attname, None)
-            if not isinstance(value, bool) and not value:
+            if not isinstance(value, (bool, int)) and not value:
                 continue
 
+            choices = getattr(f, 'choices', []) or []
+            for c_value, c_label in choices:
+                if c_value == value:
+                    value = c_label
+                    break
+
             if getattr(f, 'primary_key', False):
                 f.verbose_name = 'id'
             elif isinstance(value, list):

apps/authentication/api/sso.py

@@ -14,7 +14,6 @@ from rest_framework.response import Response
 from authentication.errors import ACLError
 from common.api import JMSGenericViewSet
 from common.const.http import POST, GET
-from common.permissions import OnlySuperUser
 from common.serializers import EmptySerializer
 from common.utils import reverse, safe_next_url
 from common.utils.timezone import utc_now
@@ -38,8 +37,11 @@ class SSOViewSet(AuthMixin, JMSGenericViewSet):
         'login_url': SSOTokenSerializer,
         'login': EmptySerializer
     }
-
-    @action(methods=[POST], detail=False, permission_classes=[OnlySuperUser], url_path='login-url')
+    rbac_perms = {
+        'login_url': 'authentication.add_ssotoken',
+    }
+    
+    @action(methods=[POST], detail=False, url_path='login-url')
     def login_url(self, request, *args, **kwargs):
         if not settings.AUTH_SSO:
             raise SSOAuthClosed()
@@ -103,11 +105,9 @@ class SSOViewSet(AuthMixin, JMSGenericViewSet):
             self.request.session['auth_backend'] = settings.AUTH_BACKEND_SSO
             login(self.request, user, settings.AUTH_BACKEND_SSO)
             self.send_auth_signal(success=True, user=user)
-            self.mark_mfa_ok('otp', user)
 
             LoginIpBlockUtil(ip).clean_block_if_need()
             LoginBlockUtil(username, ip).clean_failed_count()
-            self.clear_auth_mark()
         except (ACLError, LoginConfirmBaseError):  # 无需记录日志
             pass
         except (AuthFailedError, SSOAuthKeyTTLError) as e:

apps/authentication/backends/oauth2/backends.py

@@ -1,5 +1,6 @@
 # -*- coding: utf-8 -*-
 #
+import base64
 import requests
 
 from django.utils.translation import gettext_lazy as _
@@ -67,14 +68,6 @@ class OAuth2Backend(JMSModelBackend):
             response_data = response_data['data']
         return response_data
 
-    @staticmethod
-    def get_query_dict(response_data, query_dict):
-        query_dict.update({
-            'uid': response_data.get('uid', ''),
-            'access_token': response_data.get('access_token', '')
-        })
-        return query_dict
-
     def authenticate(self, request, code=None, **kwargs):
         log_prompt = "Process authenticate [OAuth2Backend]: {}"
         logger.debug(log_prompt.format('Start'))
@@ -83,29 +76,31 @@ class OAuth2Backend(JMSModelBackend):
             return None
 
         query_dict = {
-            'client_id': settings.AUTH_OAUTH2_CLIENT_ID,
-            'client_secret': settings.AUTH_OAUTH2_CLIENT_SECRET,
-            'grant_type': 'authorization_code',
-            'code': code,
+            'grant_type': 'authorization_code', 'code': code,
             'redirect_uri': build_absolute_uri(
                 request, path=reverse(settings.AUTH_OAUTH2_AUTH_LOGIN_CALLBACK_URL_NAME)
             )
         }
-        if '?' in settings.AUTH_OAUTH2_ACCESS_TOKEN_ENDPOINT:
-            separator = '&'
-        else:
-            separator = '?'
+        separator = '&' if '?' in settings.AUTH_OAUTH2_ACCESS_TOKEN_ENDPOINT else '?'
         access_token_url = '{url}{separator}{query}'.format(
-            url=settings.AUTH_OAUTH2_ACCESS_TOKEN_ENDPOINT, separator=separator, query=urlencode(query_dict)
+            url=settings.AUTH_OAUTH2_ACCESS_TOKEN_ENDPOINT,
+            separator=separator, query=urlencode(query_dict)
         )
         # token_method -> get, post(post_data), post_json
         token_method = settings.AUTH_OAUTH2_ACCESS_TOKEN_METHOD.lower()
         logger.debug(log_prompt.format('Call the access token endpoint[method: %s]' % token_method))
+        encoded_credentials = base64.b64encode(
+            f"{settings.AUTH_OAUTH2_CLIENT_ID}:{settings.AUTH_OAUTH2_CLIENT_SECRET}".encode()
+        ).decode()
         headers = {
-            'Accept': 'application/json'
+            'Accept': 'application/json', 'Authorization': f'Basic {encoded_credentials}'
         }
         if token_method.startswith('post'):
             body_key = 'json' if token_method.endswith('json') else 'data'
+            query_dict.update({
+                'client_id': settings.AUTH_OAUTH2_CLIENT_ID,
+                'client_secret': settings.AUTH_OAUTH2_CLIENT_SECRET,
+            })
             access_token_response = requests.post(
                 access_token_url, headers=headers, **{body_key: query_dict}
             )
@@ -121,22 +116,12 @@ class OAuth2Backend(JMSModelBackend):
             logger.error(log_prompt.format(error))
             return None
 
-        query_dict = self.get_query_dict(response_data, query_dict)
-
         headers = {
             'Accept': 'application/json',
             'Authorization': 'Bearer {}'.format(response_data.get('access_token', ''))
         }
-
         logger.debug(log_prompt.format('Get userinfo endpoint'))
-        if '?' in settings.AUTH_OAUTH2_PROVIDER_USERINFO_ENDPOINT:
-            separator = '&'
-        else:
-            separator = '?'
-        userinfo_url = '{url}{separator}{query}'.format(
-            url=settings.AUTH_OAUTH2_PROVIDER_USERINFO_ENDPOINT, separator=separator,
-            query=urlencode(query_dict)
-        )
+        userinfo_url = settings.AUTH_OAUTH2_PROVIDER_USERINFO_ENDPOINT
         userinfo_response = requests.get(userinfo_url, headers=headers)
         try:
             userinfo_response.raise_for_status()

apps/authentication/backends/oauth2/views.py

@@ -4,7 +4,6 @@ from django.contrib import auth
 from django.http import HttpResponseRedirect
 from django.urls import reverse
 from django.utils.http import urlencode
-from django.utils.translation import gettext_lazy as _
 
 from authentication.utils import build_absolute_uri
 from authentication.views.mixins import FlashMessageMixin
@@ -55,11 +54,7 @@ class OAuth2AuthCallbackView(View, FlashMessageMixin):
             logger.debug(log_prompt.format('Process authenticate'))
             user = authenticate(code=callback_params['code'], request=request)
 
-            if err_msg := getattr(request, 'error_message', ''):
-                login_url = reverse('authentication:login') + '?admin=1'
-                return self.get_failed_response(login_url, title=_('Authentication failed'), msg=err_msg)
-
-            if user and user.is_valid:
+            if user:
                 logger.debug(log_prompt.format('Login: {}'.format(user)))
                 auth.login(self.request, user)
                 logger.debug(log_prompt.format('Redirect'))
@@ -68,8 +63,7 @@ class OAuth2AuthCallbackView(View, FlashMessageMixin):
                 )
 
         logger.debug(log_prompt.format('Redirect'))
-        # OAuth2 服务端认证成功, 但是用户被禁用了, 这时候需要调用服务端的logout
-        redirect_url = settings.AUTH_OAUTH2_PROVIDER_END_SESSION_ENDPOINT
+        redirect_url = settings.AUTH_OAUTH2_PROVIDER_END_SESSION_ENDPOINT or '/'
         return HttpResponseRedirect(redirect_url)
 
 

apps/authentication/backends/oidc/backends.py

@@ -107,7 +107,7 @@ class OIDCAuthCodeBackend(OIDCBaseBackend):
         # parameters because we won't be able to get a valid token for the user in that case.
         if (state is None and settings.AUTH_OPENID_USE_STATE) or code is None:
             logger.debug(log_prompt.format('Authorization code or state value is missing'))
-            raise SuspiciousOperation('Authorization code or state value is missing')
+            return
 
         # Prepares the token payload that will be used to request an authentication token to the
         # token endpoint of the OIDC provider.
@@ -165,7 +165,7 @@ class OIDCAuthCodeBackend(OIDCBaseBackend):
             error = "Json token response error, token response " \
                     "content is: {}, error is: {}".format(token_response.content, str(e))
             logger.debug(log_prompt.format(error))
-            raise ParseError(error)
+            return
 
         # Validates the token.
         logger.debug(log_prompt.format('Validate ID Token'))
@@ -206,7 +206,7 @@ class OIDCAuthCodeBackend(OIDCBaseBackend):
                 error = "Json claims response error, claims response " \
                         "content is: {}, error is: {}".format(claims_response.content, str(e))
                 logger.debug(log_prompt.format(error))
-                raise ParseError(error)
+                return
 
         logger.debug(log_prompt.format('Get or create user from claims'))
         user, created = self.get_or_create_user_from_claims(request, claims)

apps/authentication/backends/saml2/views.py

@@ -1,14 +1,12 @@
 import copy
-
 from urllib import parse
 
-from django.views import View
-from django.contrib import auth
-from django.urls import reverse
 from django.conf import settings
-from django.views.decorators.csrf import csrf_exempt
+from django.contrib import auth
 from django.http import HttpResponseRedirect, HttpResponse, HttpResponseServerError
-
+from django.urls import reverse
+from django.views import View
+from django.views.decorators.csrf import csrf_exempt
 from onelogin.saml2.auth import OneLogin_Saml2_Auth
 from onelogin.saml2.errors import OneLogin_Saml2_Error
 from onelogin.saml2.idp_metadata_parser import (
@@ -16,23 +14,29 @@ from onelogin.saml2.idp_metadata_parser import (
     dict_deep_merge
 )
 
-from .settings import JmsSaml2Settings
-
 from common.utils import get_logger
+from .settings import JmsSaml2Settings
 
 logger = get_logger(__file__)
 
 
 class PrepareRequestMixin:
-    @staticmethod
-    def is_secure():
-        url_result = parse.urlparse(settings.SITE_URL)
-        return 'on' if url_result.scheme == 'https' else 'off'
+
+    @property
+    def parsed_url(self):
+        return parse.urlparse(settings.SITE_URL)
+
+    def is_secure(self):
+        return 'on' if self.parsed_url.scheme == 'https' else 'off'
+
+    def http_host(self):
+        return f"{self.parsed_url.hostname}:{self.parsed_url.port}" \
+            if self.parsed_url.port else self.parsed_url.hostname
 
     def prepare_django_request(self, request):
         result = {
             'https': self.is_secure(),
-            'http_host': request.META['HTTP_HOST'],
+            'http_host': self.http_host(),
             'script_name': request.META['PATH_INFO'],
             'get_data': request.GET.copy(),
             'post_data': request.POST.copy()
@@ -275,7 +279,7 @@ class Saml2AuthCallbackView(View, PrepareRequestMixin):
         logger.debug(log_prompt.format('Redirect'))
         redir = post_data.get('RelayState')
         if not redir or len(redir) == 0:
-          redir = "/"
+            redir = "/"
         next_url = saml_instance.redirect_to(redir)
         return HttpResponseRedirect(next_url)
 

apps/authentication/middleware.py

@@ -2,6 +2,7 @@ import base64
 
 from django.conf import settings
 from django.contrib.auth import logout as auth_logout
+from django.core.cache import cache
 from django.http import HttpResponse
 from django.shortcuts import redirect, reverse, render
 from django.utils.deprecation import MiddlewareMixin
@@ -116,23 +117,43 @@ class ThirdPartyLoginMiddleware(mixins.AuthMixin):
 
 
 class SessionCookieMiddleware(MiddlewareMixin):
+    USER_LOGIN_ENCRYPTION_KEY_PAIR = 'user_login_encryption_key_pair'
 
-    @staticmethod
-    def set_cookie_public_key(request, response):
+    def set_cookie_public_key(self, request, response):
         if request.path.startswith('/api'):
             return
-        pub_key_name = settings.SESSION_RSA_PUBLIC_KEY_NAME
-        public_key = request.session.get(pub_key_name)
-        cookie_key = request.COOKIES.get(pub_key_name)
-        if public_key and public_key == cookie_key:
+
+        session_public_key_name = settings.SESSION_RSA_PUBLIC_KEY_NAME
+        session_private_key_name = settings.SESSION_RSA_PRIVATE_KEY_NAME
+
+        session_public_key = request.session.get(session_public_key_name)
+        cookie_public_key = request.COOKIES.get(session_public_key_name)
+
+        if session_public_key and session_public_key == cookie_public_key:
             return
 
-        pri_key_name = settings.SESSION_RSA_PRIVATE_KEY_NAME
-        private_key, public_key = gen_key_pair()
+        private_key, public_key = self.get_key_pair()
+
         public_key_decode = base64.b64encode(public_key.encode()).decode()
-        request.session[pub_key_name] = public_key_decode
-        request.session[pri_key_name] = private_key
-        response.set_cookie(pub_key_name, public_key_decode)
+
+        request.session[session_public_key_name] = public_key_decode
+        request.session[session_private_key_name] = private_key
+        response.set_cookie(session_public_key_name, public_key_decode)
+
+    def get_key_pair(self):
+        key_pair = cache.get(self.USER_LOGIN_ENCRYPTION_KEY_PAIR)
+        if key_pair:
+            return key_pair['private_key'], key_pair['public_key']
+
+        private_key, public_key = gen_key_pair()
+
+        key_pair = {
+            'private_key': private_key,
+            'public_key': public_key
+        }
+        cache.set(self.USER_LOGIN_ENCRYPTION_KEY_PAIR, key_pair, None)
+
+        return private_key, public_key
 
     @staticmethod
     def set_cookie_session_prefix(request, response):

apps/authentication/mixins.py

@@ -319,20 +319,26 @@ class AuthPostCheckMixin:
 
     @classmethod
     def _check_passwd_is_too_simple(cls, user: User, password):
-        if user.is_superuser and password == 'admin':
+        if not user.is_auth_backend_model():
+            return
+        if user.check_passwd_too_simple(password):
             message = _('Your password is too simple, please change it for security')
             url = cls.generate_reset_password_url_with_flash_msg(user, message=message)
             raise errors.PasswordTooSimple(url)
 
     @classmethod
     def _check_passwd_need_update(cls, user: User):
-        if user.need_update_password:
+        if not user.is_auth_backend_model():
+            return
+        if user.check_need_update_password():
             message = _('You should to change your password before login')
             url = cls.generate_reset_password_url_with_flash_msg(user, message)
             raise errors.PasswordNeedUpdate(url)
 
     @classmethod
     def _check_password_require_reset_or_not(cls, user: User):
+        if not user.is_auth_backend_model():
+            return
         if user.password_has_expired:
             message = _('Your password has expired, please reset before logging in')
             url = cls.generate_reset_password_url_with_flash_msg(user, message)

apps/authentication/models/__init__.py

@@ -3,3 +3,4 @@ from .connection_token import *
 from .private_token import *
 from .sso_token import *
 from .temp_token import *
+from ..backends.passkey.models import *

apps/authentication/models/connection_token.py

@@ -200,7 +200,7 @@ class ConnectionToken(JMSOrgBaseModel):
 
         host_account = applet.select_host_account(self.user, self.asset)
         if not host_account:
-            raise JMSException({'error': 'No host account available'})
+            raise JMSException({'error': 'No host account available, please check the applet, host and account'})
 
         host, account, lock_key = bulk_get(host_account, ('host', 'account', 'lock_key'))
         gateway = host.domain.select_gateway() if host.domain else None

apps/authentication/signal_handlers.py

@@ -1,5 +1,5 @@
 from django.conf import settings
-from django.contrib.auth import user_logged_in
+from django.contrib.auth import user_logged_in, BACKEND_SESSION_KEY
 from django.core.cache import cache
 from django.dispatch import receiver
 from django_cas_ng.signals import cas_user_authenticated
@@ -20,8 +20,9 @@ def on_user_auth_login_success(sender, user, request, **kwargs):
             and user.mfa_enabled \
             and not request.session.get('auth_mfa'):
         request.session['auth_mfa_required'] = 1
+    auth_backend = request.session.get('auth_backend', request.session.get(BACKEND_SESSION_KEY))
     if not request.session.get("auth_third_party_done") and \
-            request.session.get('auth_backend') in AUTHENTICATION_BACKENDS_THIRD_PARTY:
+            auth_backend in AUTHENTICATION_BACKENDS_THIRD_PARTY:
         request.session['auth_third_party_required'] = 1
 
     user_session_id = request.session.get('user_session_id')

apps/authentication/templates/authentication/_msg_different_city.html

@@ -8,10 +8,8 @@
 <p>
     <b>{% trans 'Username' %}:</b> {{ username }}<br>
     <b>{% trans 'Login time' %}:</b> {{ time }}<br>
-    <b>{% trans 'Login city' %}:</b> {{ city }}({{ ip }})
+    <b>{% trans 'Login city' %}:</b> {{ city }}({{ ip }})<br>
 </p>
-
--
 <p>
     {% trans 'If you suspect that the login behavior is abnormal, please modify the account password in time.' %}
 </p>

apps/authentication/templates/authentication/_msg_reset_password.html

@@ -10,8 +10,7 @@
         {% trans 'Click here reset password' %}
     </a>
 </p>
-
--
+<br>
 <p>
     {% trans 'This link is valid for 1 hour. After it expires' %}
     <a href="{{ forget_password_url }}?email={{ user.email }}">{% trans 'request new one' %}</a>

apps/authentication/templates/authentication/_msg_rest_password_success.html

@@ -5,11 +5,10 @@
     {% trans 'Your password has just been successfully updated' %}
 </p>
 <p>
-    <b>{% trans 'IP' %}:</b> {{ ip_address }} <br />
-    <b>{% trans 'Browser' %}:</b> {{ browser }}
+    <b>{% trans 'IP' %}:</b> {{ ip_address }} <br/>
+    <b>{% trans 'Browser' %}:</b> {{ browser }} <br>
 </p>
--
 <p>
-    {% trans 'If the password update was not initiated by you, your account may have security issues' %} <br />
+    {% trans 'If the password update was not initiated by you, your account may have security issues' %} <br/>
     {% trans 'If you have any questions, you can contact the administrator' %}
 </p>

apps/authentication/templates/authentication/_msg_rest_public_key_success.html

@@ -5,11 +5,10 @@
     {% trans 'Your public key has just been successfully updated' %}
 </p>
 <p>
-    <b>{% trans 'IP' %}:</b> {{ ip_address }} <br />
-    <b>{% trans 'Browser' %}:</b> {{ browser }}
+    <b>{% trans 'IP' %}:</b> {{ ip_address }} <br>
+    <b>{% trans 'Browser' %}:</b> {{ browser }}<br>
 </p>
--
 <p>
-    {% trans 'If the public key update was not initiated by you, your account may have security issues' %} <br />
+    {% trans 'If the public key update was not initiated by you, your account may have security issues' %} <br/>
     {% trans 'If you have any questions, you can contact the administrator' %}
 </p>

apps/authentication/templates/authentication/login.html

@@ -396,7 +396,7 @@
 
 </body>
 {% include '_foot_js.html' %}
-<script type="text/javascript" src="/static/js/plugins/jsencrypt/jsencrypt.min.js"></script>
+<script type="text/javascript" src="/static/js/plugins/jsencrypt/jsencrypt.3.3.2.min.js"></script>
 <script type="text/javascript" src="/static/js/plugins/cryptojs/crypto-js.min.js"></script>
 <script type="text/javascript" src="/static/js/plugins/buffer/buffer.min.js"></script>
 <script>

apps/authentication/tests.py

@@ -1,4 +1,4 @@
-from .utils import gen_key_pair, rsa_decrypt, rsa_encrypt
+from common.utils import gen_key_pair, rsa_decrypt, rsa_encrypt
 
 
 def test_rsa_encrypt_decrypt(message='test-password-$%^&*'):

apps/authentication/views/base.py

@@ -46,9 +46,6 @@ class BaseLoginCallbackView(AuthMixin, FlashMessageMixin, IMClientMixin, View):
     def verify_state(self):
         raise NotImplementedError
 
-    def get_verify_state_failed_response(self, redirect_uri):
-        raise NotImplementedError
-
     def create_user_if_not_exist(self, user_id, **kwargs):
         user = None
         user_attr = self.client.get_user_detail(user_id, **kwargs)
@@ -122,9 +119,6 @@ class BaseBindCallbackView(FlashMessageMixin, IMClientMixin, View):
     def verify_state(self):
         raise NotImplementedError
 
-    def get_verify_state_failed_response(self, redirect_uri):
-        raise NotImplementedError
-
     def get_already_bound_response(self, redirect_uri):
         raise NotImplementedError
 
@@ -151,11 +145,9 @@ class BaseBindCallbackView(FlashMessageMixin, IMClientMixin, View):
             setattr(user, f'{self.auth_type}_id', auth_user_id)
             user.save()
         except IntegrityError as e:
-            if e.args[0] == 1062:
-                msg = _('The %s is already bound to another user') % self.auth_type_label
-                response = self.get_failed_response(redirect_url, msg, msg)
-                return response
-            raise e
+            msg = _('The %s is already bound to another user') % self.auth_type_label
+            response = self.get_failed_response(redirect_url, msg, msg)
+            return response
 
         ip = get_request_ip(request)
         OAuthBindMessage(user, ip, self.auth_type_label, auth_user_id).publish_async()

apps/authentication/views/dingtalk.py

@@ -47,15 +47,7 @@ class DingTalkBaseMixin(UserConfirmRequiredExceptionMixin, PermissionsMixin, Fla
             )
 
     def verify_state(self):
-        state = self.request.GET.get('state')
-        session_state = self.request.session.get(DINGTALK_STATE_SESSION_KEY)
-        if state != session_state:
-            return False
-        return True
-
-    def get_verify_state_failed_response(self, redirect_uri):
-        msg = _("The system configuration is incorrect. Please contact your administrator")
-        return self.get_failed_response(redirect_uri, msg, msg)
+        return self.verify_state_with_session_key(DINGTALK_STATE_SESSION_KEY)
 
     def get_already_bound_response(self, redirect_url):
         msg = _('DingTalk is already bound')

apps/authentication/views/feishu.py

@@ -58,15 +58,7 @@ class FeiShuQRMixin(UserConfirmRequiredExceptionMixin, PermissionsMixin, FlashMe
             )
 
     def verify_state(self):
-        state = self.request.GET.get('state')
-        session_state = self.request.session.get(self.state_session_key)
-        if state != session_state:
-            return False
-        return True
-
-    def get_verify_state_failed_response(self, redirect_uri):
-        msg = _("The system configuration is incorrect. Please contact your administrator")
-        return self.get_failed_response(redirect_uri, msg, msg)
+        return self.verify_state_with_session_key(self.state_session_key)
 
     def get_qr_url(self, redirect_uri):
         state = random_string(16)

apps/authentication/views/login.py

@@ -249,6 +249,8 @@ class UserLoginView(mixins.AuthMixin, UserLoginContextMixin, FormView):
     def form_valid(self, form):
         if not self.request.session.test_cookie_worked():
             form.add_error(None, _("Login timeout, please try again."))
+            # 当 session 过期后，刷新浏览器重新提交依旧会报错，所以需要重新设置 test_cookie
+            self.request.session.set_test_cookie()
             return self.form_invalid(form)
 
         # https://docs.djangoproject.com/en/3.1/topics/http/sessions/#setting-test-cookies

apps/authentication/views/mixins.py

@@ -1,5 +1,6 @@
 # -*- coding: utf-8 -*-
 #
+from django.utils.translation import gettext_lazy as _
 from common.utils import FlashMessageUtil
 
 
@@ -32,3 +33,12 @@ class FlashMessageMixin:
 
     def get_failed_response(self, redirect_url, title, msg, interval=10):
         return self.get_response(redirect_url, title, msg, 'error', interval)
+
+    def get_verify_state_failed_response(self, redirect_uri):
+        msg = _(
+            "For your safety, automatic redirection login is not supported on the client."
+            " If you need to open it in the client, please log in again")
+        return self.get_failed_response(redirect_uri, msg, msg)
+
+    def verify_state_with_session_key(self, session_key):
+        return self.request.GET.get('state') == self.request.session.get(session_key)

apps/authentication/views/slack.py

@@ -37,15 +37,7 @@ class SlackMixin(UserConfirmRequiredExceptionMixin, PermissionsMixin, FlashMessa
             )
 
     def verify_state(self):
-        state = self.request.GET.get('state')
-        session_state = self.request.session.get(SLACK_STATE_SESSION_KEY)
-        if state != session_state:
-            return False
-        return True
-
-    def get_verify_state_failed_response(self, redirect_uri):
-        msg = _("The system configuration is incorrect. Please contact your administrator")
-        return self.get_failed_response(redirect_uri, msg, msg)
+        return self.verify_state_with_session_key(SLACK_STATE_SESSION_KEY)
 
     def get_qr_url(self, redirect_uri):
         state = random_string(16)

apps/authentication/views/wecom.py

@@ -45,15 +45,7 @@ class WeComBaseMixin(UserConfirmRequiredExceptionMixin, PermissionsMixin, FlashM
             )
 
     def verify_state(self):
-        state = self.request.GET.get('state')
-        session_state = self.request.session.get(WECOM_STATE_SESSION_KEY)
-        if state != session_state:
-            return False
-        return True
-
-    def get_verify_state_failed_response(self, redirect_uri):
-        msg = _("The system configuration is incorrect. Please contact your administrator")
-        return self.get_failed_response(redirect_uri, msg, msg)
+        return self.verify_state_with_session_key(WECOM_STATE_SESSION_KEY)
 
     def get_already_bound_response(self, redirect_url):
         msg = _('WeCom is already bound')

apps/common/api/action.py

@@ -8,6 +8,8 @@ from rest_framework.request import Request
 from rest_framework.response import Response
 
 from common.const.http import POST
+from orgs.models import Organization
+from orgs.utils import current_org
 
 __all__ = ['SuggestionMixin', 'RenderToJsonMixin']
 
@@ -23,7 +25,16 @@ class SuggestionMixin:
 
     @action(methods=['get'], detail=False, url_path='suggestions')
     def match(self, request, *args, **kwargs):
-        queryset = self.filter_queryset(self.get_queryset())
+        queryset = self.get_queryset()
+        org_id = str(current_org.id)
+        if (
+                not request.user.is_superuser and
+                org_id != Organization.ROOT_ID and
+                not request.user.orgs.filter(id=org_id).exists()
+        ):
+            queryset = queryset.none()
+
+        queryset = self.filter_queryset(queryset)
         queryset = queryset[:self.suggestion_limit]
         page = self.paginate_queryset(queryset)
 

apps/common/const/signals.py

@@ -16,4 +16,4 @@ POST_CLEAR = 'post_clear'
 POST_PREFIX = 'post'
 PRE_PREFIX = 'pre'
 
-SKIP_SIGNAL = 'skip_signal'
+OP_LOG_SKIP_SIGNAL = 'operate_log_skip_signal'

apps/common/db/models.py

@@ -17,7 +17,7 @@ from django.db.models import F, ExpressionWrapper, CASCADE
 from django.db.models import QuerySet
 from django.utils.translation import gettext_lazy as _
 
-from ..const.signals import SKIP_SIGNAL
+from ..const.signals import OP_LOG_SKIP_SIGNAL
 
 
 class ChoicesMixin:
@@ -82,7 +82,7 @@ def CASCADE_SIGNAL_SKIP(collector, field, sub_objs, using):
     # 级联删除时，操作日志标记不保存，以免用户混淆
     try:
         for obj in sub_objs:
-            setattr(obj, SKIP_SIGNAL, True)
+            setattr(obj, OP_LOG_SKIP_SIGNAL, True)
     except:
         pass
 

apps/common/management/commands/check_api.py

@@ -70,7 +70,8 @@ known_unauth_urls = [
     "/api/v1/authentication/login-confirm-ticket/status/",
     "/api/v1/authentication/mfa/select/",
     "/api/v1/authentication/mfa/send-code/",
-    "/api/v1/authentication/sso/login/"
+    "/api/v1/authentication/sso/login/",
+    "/api/v1/authentication/user-session/"
 ]
 
 known_error_urls = [
@@ -91,7 +92,6 @@ class Command(BaseCommand):
         unauth_urls = []
         error_urls = []
         unformat_urls = []
-
         for url, ourl in urls:
             if '(' in url or '<' in url:
                 unformat_urls.append([url, ourl])

apps/common/plugins/es.py

@@ -14,9 +14,13 @@ from uuid import UUID
 
 from django.utils.translation import gettext_lazy as _
 from django.db.models import QuerySet as DJQuerySet
-from elasticsearch import Elasticsearch
-from elasticsearch.helpers import bulk
-from elasticsearch.exceptions import RequestError, NotFoundError
+from elasticsearch7 import Elasticsearch
+from elasticsearch7.helpers import bulk
+from elasticsearch7.exceptions import RequestError, SSLError
+from elasticsearch7.exceptions import NotFoundError as NotFoundError7
+
+from elasticsearch8.exceptions import NotFoundError as NotFoundError8
+from elasticsearch8.exceptions import BadRequestError
 
 from common.utils.common import lazyproperty
 from common.utils import get_logger
@@ -36,9 +40,82 @@ class NotSupportElasticsearch8(JMSException):
     default_detail = _('Not Support Elasticsearch8')
 
 
-class ES(object):
-    def __init__(self, config, properties, keyword_fields, exact_fields=None, match_fields=None):
+class InvalidElasticsearchSSL(JMSException):
+    default_code = 'invalid_elasticsearch_SSL'
+    default_detail = _(
+        'Connection failed: Self-signed certificate used. Please check server certificate configuration')
 
+
+class ESClient(object):
+
+    def __new__(cls, *args, **kwargs):
+        version = get_es_client_version(**kwargs)
+        if version == 6:
+            return ESClientV6(*args, **kwargs)
+        if version == 7:
+            return ESClientV7(*args, **kwargs)
+        elif version == 8:
+            return ESClientV8(*args, **kwargs)
+        raise ValueError('Unsupported ES_VERSION %r' % version)
+
+
+class ESClientBase(object):
+    @classmethod
+    def get_properties(cls, data, index):
+        return data[index]['mappings']['properties']
+
+    @classmethod
+    def get_mapping(cls, properties):
+        return {'mappings': {'properties': properties}}
+
+
+class ESClientV7(ESClientBase):
+    def __init__(self, *args, **kwargs):
+        from elasticsearch7 import Elasticsearch
+        self.es = Elasticsearch(*args, **kwargs)
+
+    @classmethod
+    def get_sort(cls, field, direction):
+        return f'{field}:{direction}'
+
+
+class ESClientV6(ESClientV7):
+
+    @classmethod
+    def get_properties(cls, data, index):
+        return data[index]['mappings']['data']['properties']
+
+    @classmethod
+    def get_mapping(cls, properties):
+        return {'mappings': {'data': {'properties': properties}}}
+
+
+class ESClientV8(ESClientBase):
+    def __init__(self, *args, **kwargs):
+        from elasticsearch8 import Elasticsearch
+        self.es = Elasticsearch(*args, **kwargs)
+
+    @classmethod
+    def get_sort(cls, field, direction):
+        return {field: {'order': direction}}
+
+
+def get_es_client_version(**kwargs):
+    try:
+        es = Elasticsearch(**kwargs)
+        info = es.info()
+        version = int(info['version']['number'].split('.')[0])
+        return version
+    except SSLError:
+        raise InvalidElasticsearchSSL
+    except Exception:
+        raise InvalidElasticsearch
+
+
+class ES(object):
+
+    def __init__(self, config, properties, keyword_fields, exact_fields=None, match_fields=None):
+        self.version = 7
         self.config = config
         hosts = self.config.get('HOSTS')
         kwargs = self.config.get('OTHER', {})
@@ -46,7 +123,8 @@ class ES(object):
         ignore_verify_certs = kwargs.pop('IGNORE_VERIFY_CERTS', False)
         if ignore_verify_certs:
             kwargs['verify_certs'] = None
-        self.es = Elasticsearch(hosts=hosts, max_retries=0, **kwargs)
+        self.client = ESClient(hosts=hosts, max_retries=0, **kwargs)
+        self.es = self.client.es
         self.index_prefix = self.config.get('INDEX') or 'jumpserver'
         self.is_index_by_date = bool(self.config.get('INDEX_BY_DATE', False))
 
@@ -83,26 +161,14 @@ class ES(object):
         if not self.ping(timeout=2):
             return False
 
-        info = self.es.info()
-        version = info['version']['number'].split('.')[0]
-
-        if version == '8':
-            raise NotSupportElasticsearch8
-
         try:
             # 获取索引信息，如果没有定义，直接返回
             data = self.es.indices.get_mapping(index=self.index)
-        except NotFoundError:
+        except (NotFoundError8, NotFoundError7):
             return False
 
         try:
-            if version == '6':
-                # 检测索引是不是新的类型 es6
-                properties = data[self.index]['mappings']['data']['properties']
-            else:
-                # 检测索引是不是新的类型 es7 default index type: _doc
-                properties = data[self.index]['mappings']['properties']
-
+            properties = self.client.get_properties(data=data, index=self.index)
             for keyword in self.keyword_fields:
                 if not properties[keyword]['type'] == 'keyword':
                     break
@@ -118,21 +184,17 @@ class ES(object):
 
     def _ensure_index_exists(self):
         try:
-            info = self.es.info()
-            version = info['version']['number'].split('.')[0]
-            if version == '6':
-                mappings = {'mappings': {'data': {'properties': self.properties}}}
-            else:
-                mappings = {'mappings': {'properties': self.properties}}
+            mappings = self.client.get_mapping(self.properties)
 
             if self.is_index_by_date:
                 mappings['aliases'] = {
                     self.query_index: {}
                 }
-
+            if self.es.indices.exists(index=self.index):
+                return
             try:
                 self.es.indices.create(index=self.index, body=mappings)
-            except RequestError as e:
+            except (RequestError, BadRequestError) as e:
                 if e.error == 'resource_already_exists_exception':
                     logger.warning(e)
                 else:
@@ -175,11 +237,15 @@ class ES(object):
 
     def _filter(self, query: dict, from_=None, size=None, sort=None):
         body = self.get_query_body(**query)
-
-        data = self.es.search(
-            index=self.query_index, body=body,
-            from_=from_, size=size, sort=sort
-        )
+        search_params = {
+            'index': self.query_index,
+            'body': body,
+            'from_': from_,
+            'size': size
+        }
+        if sort is not None:
+            search_params['sort'] = sort
+        data = self.es.search(**search_params)
 
         source_data = []
         for item in data['hits']['hits']:
@@ -367,7 +433,7 @@ class QuerySet(DJQuerySet):
                     else:
                         direction = 'asc'
                     field = field.lstrip('-+')
-                    sort = f'{field}:{direction}'
+                    sort = self._storage.client.get_sort(field, direction)
                     return sort
 
     def __execute(self):

apps/common/sdk/gm/piico/cipher.py

@@ -1,6 +1,7 @@
 cipher_alg_id = {
     "sm4_ebc": 0x00000401,
     "sm4_cbc": 0x00000402,
+    "sm4_mac": 0x00000405,
 }
 
 

apps/common/sdk/sms/cmpp2.py

@@ -75,7 +75,7 @@ class CMPPSubmitRequestInstance(CMPPBaseRequestInstance):
         pk_number = struct.pack('!B', 1)
         registered_delivery = struct.pack('!B', 0)
         msg_level = struct.pack('!B', 0)
-        service_id = ((10 - len(service_id)) * '\x00' + service_id).encode('utf-8')
+        service_id = service_id.ljust(10, '\x00').encode('utf-8')
         fee_user_type = struct.pack('!B', 2)
         fee_terminal_id = ('0' * 21).encode('utf-8')
         tp_pid = struct.pack('!B', 0)
@@ -85,7 +85,7 @@ class CMPPSubmitRequestInstance(CMPPBaseRequestInstance):
         fee_code = '000000'.encode('utf-8')
         valid_time = ('\x00' * 17).encode('utf-8')
         at_time = ('\x00' * 17).encode('utf-8')
-        src_id = ((21 - len(src_id)) * '\x00' + src_id).encode('utf-8')
+        src_id = src_id.ljust(21, '\x00').encode('utf-8')
         reserve = b'\x00' * 8
         _msg_length = struct.pack('!B', len(msg_content) * 2)
         _msg_src = msg_src.encode('utf-8')

apps/common/sdk/sms/custom.py

@@ -39,8 +39,7 @@ class CustomSMS(BaseSMSClient):
             kwargs = {'params': params}
         try:
             response = action(url=settings.CUSTOM_SMS_URL, verify=False, **kwargs)
-            if response.reason != 'OK':
-                raise JMSException(detail=response.text, code=response.status_code)
+            response.raise_for_status()
         except Exception as exc:
             logger.error('Custom sms error: {}'.format(exc))
             raise JMSException(exc)

apps/common/utils/django.py

@@ -51,6 +51,19 @@ def date_expired_default():
         years = 70
     return timezone.now() + timezone.timedelta(days=365 * years)
 
+def user_date_expired_default():
+    try:
+        days = int(settings.USER_DEFAULT_EXPIRED_DAYS)
+    except TypeError:
+        days = 25550
+    return timezone.now() + timezone.timedelta(days=days)
+
+def asset_permission_date_expired_default():
+    try:
+        days = int(settings.ASSET_PERMISSION_DEFAULT_EXPIRED_DAYS)
+    except TypeError:
+        days = 25550
+    return timezone.now() + timezone.timedelta(days=days)
 
 def union_queryset(*args, base_queryset=None):
     if len(args) == 1:

apps/common/utils/translate.py

@@ -21,6 +21,7 @@ def i18n_fmt(tpl, *args):
         return tpl
 
     args = [str(arg) for arg in args]
+    args = [arg.replace(', ', ' ') for arg in args]
 
     try:
         tpl % tuple(args)

apps/common/utils/verify_code.py

@@ -13,9 +13,9 @@ from common.utils.random import random_string
 logger = get_logger(__file__)
 
 
-@shared_task(verbose_name=_('Send email'))
-def send_async(sender):
-    sender.gen_and_send()
+@shared_task(verbose_name=_('Send SMS code'))
+def send_sms_async(target, code):
+    SMS().send_verify_code(target, code)
 
 
 class SendAndVerifyCodeUtil(object):
@@ -35,7 +35,7 @@ class SendAndVerifyCodeUtil(object):
             logger.warning('Send sms too frequently, delay {}'.format(ttl))
             raise CodeSendTooFrequently(ttl)
 
-        return send_async.apply_async(kwargs={"sender": self}, priority=100)
+        return self.gen_and_send()
 
     def gen_and_send(self):
         try:
@@ -72,13 +72,15 @@ class SendAndVerifyCodeUtil(object):
         return code
 
     def __send_with_sms(self):
-        sms = SMS()
-        sms.send_verify_code(self.target, self.code)
+        send_sms_async.apply_async(args=(self.target, self.code), priority=100)
 
     def __send_with_email(self):
-        subject = self.other_args.get('subject')
-        message = self.other_args.get('message')
-        send_mail_async(subject, message, [self.target], html_message=message)
+        subject = self.other_args.get('subject', '')
+        message = self.other_args.get('message', '')
+        send_mail_async.apply_async(
+            args=(subject, message, [self.target]),
+            kwargs={'html_message': message}, priority=100
+        )
 
     def __send(self, code):
         """