perf: modify url

Co-authored-by: feng <1304903146@qq.com>

.github/ISSUE_TEMPLATE/----.md

@@ -1,11 +1,35 @@
 ---
 name: 需求建议
 about: 提出针对本项目的想法和建议
-title: "[Feature] "
+title: "[Feature] 需求标题"
 labels: 类型:需求
 assignees: 
   - ibuler
   - baijiangjie
 ---
 
-**请描述您的需求或者改进建议.**
+## 注意
+_针对过于简单的需求描述不予考虑。请确保提供足够的细节和信息以支持功能的开发和实现。_
+
+## 功能名称
+[在这里输入功能的名称或标题]
+
+## 功能描述
+[在这里描述该功能的详细内容，包括其作用、目的和所需的功能]
+
+## 用户故事（可选）
+[如果适用，可以提供用户故事来更好地理解该功能的使用场景和用户期望]
+
+## 功能要求
+- [要求1：描述该功能的具体要求，如界面设计、交互逻辑等]
+- [要求2：描述该功能的另一个具体要求]
+- [以此类推，列出所有相关的功能要求]
+
+## 示例或原型（可选）
+[如果有的话，提供该功能的示例或原型图以更好地说明功能的实现方式]
+
+## 优先级
+[描述该功能的优先级，如高、中、低，或使用数字等其他标识]
+
+## 备注（可选）
+[在这里添加任何其他相关信息或备注]

.github/ISSUE_TEMPLATE/bug---.md

@@ -1,22 +1,51 @@
 ---
 name: Bug 提交
 about: 提交产品缺陷帮助我们更好的改进
-title: "[Bug] "
+title: "[Bug] Bug 标题"
 labels: 类型:Bug
 assignees: 
    - baijiangjie
 ---
 
-**JumpServer 版本( v2.28 之前的版本不再支持 )**
+## 注意
+**JumpServer 版本( v2.28 之前的版本不再支持 )** <br>
+_针对过于简单的 Bug 描述不予考虑。请确保提供足够的细节和信息以支持 Bug 的复现和修复。_
+
+## 当前使用的 JumpServer 版本 (必填)
+[在这里输入当前使用的 JumpServer 的版本号]
+
+## 使用的版本类型 (必填)
+- [ ] 社区版
+- [ ] 企业版
+- [ ] 企业试用版
 
 
-**浏览器版本**
+## 版本安装方式 (必填)
+- [ ] 在线安装 (一键命令)
+- [ ] 离线安装 (下载离线包)
+- [ ] All-in-One
+- [ ] 1Panel 安装
+- [ ] Kubernetes 安装
+- [ ] 源码安装
 
+## Bug 描述 (详细)
+[在这里描述 Bug 的详细情况，包括其影响和出现的具体情况]
 
-**Bug 描述**
+## 复现步骤
+1. [描述如何复现 Bug 的第一步]
+2. [描述如何复现 Bug 的第二步]
+3. [以此类推，列出所有复现 Bug 所需的步骤]
 
+## 期望行为
+[描述 Bug 出现时期望的系统行为或结果]
 
-**Bug 重现步骤(有截图更好)**
+## 实际行为
-1.
+[描述实际上发生了什么，以及 Bug 出现的具体情况]
-2.
+
-3.
+## 系统环境
+- 操作系统：[例如：Windows 10, macOS Big Sur]
+- 浏览器/应用版本：[如果适用，请提供相关版本信息]
+- 其他相关环境信息：[如果有其他相关环境信息，请在此处提供]
+
+## 附加信息（可选）
+[在这里添加任何其他相关信息，如截图、错误信息等]

.github/ISSUE_TEMPLATE/question.md

@@ -1,10 +1,50 @@
 ---
 name: 问题咨询
 about: 提出针对本项目安装部署、使用及其他方面的相关问题
-title: "[Question] "
+title: "[Question] 问题标题"
 labels: 类型:提问
 assignees: 
   - baijiangjie
 ---
+## 注意
+**请描述您的问题.** <br>
+**JumpServer 版本( v2.28 之前的版本不再支持 )** <br>
+_针对过于简单的 Bug 描述不予考虑。请确保提供足够的细节和信息以支持 Bug 的复现和修复。_
+
+## 当前使用的 JumpServer 版本 (必填)
+[在这里输入当前使用的 JumpServer 的版本号]
+
+## 使用的版本类型 (必填)
+- [ ] 社区版
+- [ ] 企业版
+- [ ] 企业试用版
+
+
+## 版本安装方式 (必填)
+- [ ] 在线安装 (一键命令)
+- [ ] 离线安装 (下载离线包)
+- [ ] All-in-One
+- [ ] 1Panel 安装
+- [ ] Kubernetes 安装
+- [ ] 源码安装
+
+## 问题描述 (详细)
+[在这里描述你遇到的问题]
+
+## 背景信息
+- 操作系统：[例如：Windows 10, macOS Big Sur]
+- 浏览器/应用版本：[如果适用，请提供相关版本信息]
+- 其他相关环境信息：[如果有其他相关环境信息，请在此处提供]
+
+## 具体问题
+[在这里详细描述你的问题，包括任何相关细节或错误信息]
+
+## 尝试过的解决方法
+[如果你已经尝试过解决问题，请在这里列出你已经尝试过的解决方法]
+
+## 预期结果
+[描述你期望的解决方案或结果]
+
+## 我们的期望
+[描述你希望我们提供的帮助或支持]
 
-**请描述您的问题.**

.github/workflows/jms-build-test.yml

@@ -1,26 +1,32 @@
 name: "Run Build Test"
 on:
   push:
-    branches:
+    paths:
-      - pr@*
+      - 'Dockerfile'
-      - repr@*
+      - 'Dockerfile-*'
+      - 'pyproject.toml'
+      - 'poetry.lock'
 
 jobs:
   build:
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v3
+    - uses: docker/setup-qemu-action@v3
+    - uses: docker/setup-buildx-action@v3
 
-    - uses: docker/setup-qemu-action@v2
+    - name: Check Dockerfile
+      run: |
+        test -f Dockerfile-ce || cp -f Dockerfile Dockerfile-ce
 
-    - uses: docker/setup-buildx-action@v2
+    - name: Build CE Image
-
+      uses: docker/build-push-action@v5
-    - uses: docker/build-push-action@v3
       with:
         context: .
         push: false
-        tags: jumpserver/core-ce:test
         file: Dockerfile-ce
+        tags: jumpserver/core-ce:test
+        platforms: linux/amd64
         build-args: |
           APT_MIRROR=http://deb.debian.org
           PIP_MIRROR=https://pypi.org/simple
@@ -28,9 +34,22 @@ jobs:
         cache-from: type=gha
         cache-to: type=gha,mode=max
 
-    - uses: LouisBrunner/checks-action@v1.5.0
+    - name: Prepare EE Image
-      if: always()
+      run: |
+        sed -i 's@^FROM registry.fit2cloud.com@# FROM registry.fit2cloud.com@g' Dockerfile-ee
+        sed -i 's@^COPY --from=build-xpack@# COPY --from=build-xpack@g' Dockerfile-ee
+
+    - name: Build EE Image
+      uses: docker/build-push-action@v5
       with:
-        token: ${{ secrets.GITHUB_TOKEN }}
+        context: .
-        name: Check Build
+        push: false
-        conclusion: ${{ job.status }}
+        file: Dockerfile-ee
+        tags: jumpserver/core-ee:test
+        platforms: linux/amd64
+        build-args: |
+          APT_MIRROR=http://deb.debian.org
+          PIP_MIRROR=https://pypi.org/simple
+          PIP_JMS_MIRROR=https://pypi.org/simple
+        cache-from: type=gha
+        cache-to: type=gha,mode=max

.gitignore

@@ -43,3 +43,4 @@ releashe
 data/*
 test.py
 .history/
+.test/

Dockerfile-ce

@@ -19,11 +19,11 @@ ARG BUILD_DEPENDENCIES="              \
 
 ARG DEPENDENCIES="                    \
         freetds-dev                   \
-        libpq-dev                     \
         libffi-dev                    \
         libjpeg-dev                   \
         libkrb5-dev                   \
         libldap2-dev                  \
+        libpq-dev                     \
         libsasl2-dev                  \
         libssl-dev                    \
         libxml2-dev                   \
@@ -75,7 +75,9 @@ ENV LANG=zh_CN.UTF-8 \
 
 ARG DEPENDENCIES="                    \
         libjpeg-dev                   \
+        libpq-dev                     \
         libx11-dev                    \
+        freerdp2-dev                  \
         libxmlsec1-openssl"
 
 ARG TOOLS="                           \
@@ -85,6 +87,7 @@ ARG TOOLS="                           \
         default-mysql-client          \
         iputils-ping                  \
         locales                       \
+        netcat-openbsd                \
         nmap                          \
         openssh-client                \
         patch                         \
@@ -109,8 +112,17 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked,id=core-apt \
     && sed -i "s@# export @export @g" ~/.bashrc \
     && sed -i "s@# alias @alias @g" ~/.bashrc
 
+ARG RECEPTOR_VERSION=v1.4.5
+RUN set -ex \
+    && wget -O /opt/receptor.tar.gz https://github.com/ansible/receptor/releases/download/${RECEPTOR_VERSION}/receptor_${RECEPTOR_VERSION/v/}_linux_${TARGETARCH}.tar.gz \
+    && tar -xf /opt/receptor.tar.gz -C /usr/local/bin/ \
+    && chown root:root /usr/local/bin/receptor \
+    && chmod 755 /usr/local/bin/receptor \
+    && rm -f /opt/receptor.tar.gz
+
 COPY --from=stage-2 /opt/py3 /opt/py3
 COPY --from=stage-1 /opt/jumpserver/release/jumpserver /opt/jumpserver
+COPY --from=stage-1 /opt/jumpserver/release/jumpserver/apps/libs/ansible/ansible.cfg /etc/ansible/
 
 WORKDIR /opt/jumpserver
 

GITSHA

@@ -1 +0,0 @@
-a92023840a13ac81abf21d0291cba2aaee23943b

README.md

@@ -94,7 +94,8 @@ JumpServer 堡垒机帮助企业以更安全的方式管控和登录各种类型
 | [KoKo](https://github.com/jumpserver/koko)             | <a href="https://github.com/jumpserver/koko/releases"><img alt="Koko release" src="https://img.shields.io/github/release/jumpserver/koko.svg" /></a>                   | JumpServer 字符协议 Connector 项目                                                      |
 | [Lion](https://github.com/jumpserver/lion-release)     | <a href="https://github.com/jumpserver/lion-release/releases"><img alt="Lion release" src="https://img.shields.io/github/release/jumpserver/lion-release.svg" /></a>   | JumpServer 图形协议 Connector 项目，依赖 [Apache Guacamole](https://guacamole.apache.org/) |
 | [Razor](https://github.com/jumpserver/razor)           | <img alt="Chen" src="https://img.shields.io/badge/release-私有发布-red" />                                                                                                 | JumpServer RDP 代理 Connector 项目                                                    |
-| [Tinker](https://github.com/jumpserver/tinker)         | <img alt="Tinker" src="https://img.shields.io/badge/release-私有发布-red" />                                                                                               | JumpServer 远程应用 Connector 项目                                                      |
+| [Tinker](https://github.com/jumpserver/tinker)         | <img alt="Tinker" src="https://img.shields.io/badge/release-私有发布-red" />                                                                                               | JumpServer 远程应用 Connector 项目 (Windows)                                                      |
+| [Panda](https://github.com/jumpserver/Panda)         | <img alt="Panda" src="https://img.shields.io/badge/release-私有发布-red" />                                                                                               | JumpServer 远程应用 Connector 项目 (Linux)                                                      |
 | [Magnus](https://github.com/jumpserver/magnus-release) | <a href="https://github.com/jumpserver/magnus-release/releases"><img alt="Magnus release" src="https://img.shields.io/github/release/jumpserver/magnus-release.svg" /> | JumpServer 数据库代理 Connector 项目                                                     |
 | [Chen](https://github.com/jumpserver/chen-release)     | <a href="https://github.com/jumpserver/chen-release/releases"><img alt="Chen release" src="https://img.shields.io/github/release/jumpserver/chen-release.svg" />       | JumpServer Web DB 项目，替代原来的 OmniDB                                                 |
 | [Kael](https://github.com/jumpserver/kael)             | <a href="https://github.com/jumpserver/kael/releases"><img alt="Kael release" src="https://img.shields.io/github/release/jumpserver/kael.svg" />                       | JumpServer 连接 GPT 资产的组件项目                                                         |
@@ -112,7 +113,7 @@ JumpServer是一款安全产品，请参考 [基本安全建议](https://docs.ju
 
 ## License & Copyright
 
-Copyright (c) 2014-2023 飞致云 FIT2CLOUD, All rights reserved.
+Copyright (c) 2014-2024 飞致云 FIT2CLOUD, All rights reserved.
 
 Licensed under The GNU General Public License version 3 (GPLv3)  (the "License"); you may not use this file except in
 compliance with the License. You may obtain a copy of the License at

README_EN.md

@@ -85,7 +85,7 @@ If you find a security problem, please contact us directly：
 - 400-052-0755
 
 ### License & Copyright
-Copyright (c) 2014-2022 FIT2CLOUD Tech, Inc., All rights reserved.
+Copyright (c) 2014-2024 FIT2CLOUD Tech, Inc., All rights reserved.
 
 Licensed under The GNU General Public License version 3 (GPLv3)  (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
 

apps/accounts/api/account/task.py

@@ -1,9 +1,13 @@
+from django.db.models import Q
 from rest_framework.generics import CreateAPIView
-from rest_framework.response import Response
 
 from accounts import serializers
-from accounts.tasks import verify_accounts_connectivity_task, push_accounts_to_assets_task
+from accounts.models import Account
-from assets.exceptions import NotSupportedTemporarilyError
+from accounts.permissions import AccountTaskActionPermission
+from accounts.tasks import (
+    remove_accounts_task, verify_accounts_connectivity_task, push_accounts_to_assets_task
+)
+from authentication.permissions import UserConfirmation, ConfirmType
 
 __all__ = [
     'AccountsTaskCreateAPI',
@@ -12,40 +16,48 @@ __all__ = [
 
 class AccountsTaskCreateAPI(CreateAPIView):
     serializer_class = serializers.AccountTaskSerializer
+    permission_classes = (AccountTaskActionPermission,)
 
-    def check_permissions(self, request):
+    def get_permissions(self):
-        act = request.data.get('action')
+        act = self.request.data.get('action')
-        if act == 'push':
+        if act == 'remove':
-            code = 'accounts.push_account'
+            self.permission_classes = [
-        else:
+                AccountTaskActionPermission,
-            code = 'accounts.verify_account'
+                UserConfirmation.require(ConfirmType.PASSWORD)
-        has = request.user.has_perm(code)
+            ]
-        if not has:
+        return super().get_permissions()
-            self.permission_denied(request)
+
+    @staticmethod
+    def get_account_ids(data, action):
+        account_type = 'gather_accounts' if action == 'remove' else 'accounts'
+        accounts = data.get(account_type, [])
+        account_ids = [str(a.id) for a in accounts]
+
+        if action == 'remove':
+            return account_ids
+
+        assets = data.get('assets', [])
+        asset_ids = [str(a.id) for a in assets]
+        ids = Account.objects.filter(
+            Q(id__in=account_ids) | Q(asset_id__in=asset_ids)
+        ).distinct().values_list('id', flat=True)
+        return [str(_id) for _id in ids]
 
     def perform_create(self, serializer):
         data = serializer.validated_data
-        accounts = data.get('accounts', [])
+        action = data['action']
-        params = data.get('params')
+        ids = self.get_account_ids(data, action)
-        account_ids = [str(a.id) for a in accounts]
 
-        if data['action'] == 'push':
+        if action == 'push':
-            task = push_accounts_to_assets_task.delay(account_ids, params)
+            task = push_accounts_to_assets_task.delay(ids, data.get('params'))
+        elif action == 'remove':
+            task = remove_accounts_task.delay(ids)
+        elif action == 'verify':
+            task = verify_accounts_connectivity_task.delay(ids)
         else:
-            account = accounts[0]
+            raise ValueError(f"Invalid action: {action}")
-            asset = account.asset
-            if not asset.auto_config['ansible_enabled'] or \
-                    not asset.auto_config['ping_enabled']:
-                raise NotSupportedTemporarilyError()
-            task = verify_accounts_connectivity_task.delay(account_ids)
 
         data = getattr(serializer, '_data', {})
         data["task"] = task.id
         setattr(serializer, '_data', data)
         return task
-
-    def get_exception_handler(self):
-        def handler(e, context):
-            return Response({"error": str(e)}, status=401)
-
-        return handler

apps/accounts/api/automations/backup.py

@@ -18,9 +18,8 @@ __all__ = [
 
 class AccountBackupPlanViewSet(OrgBulkModelViewSet):
     model = AccountBackupAutomation
-    filter_fields = ('name',)
+    filterset_fields = ('name',)
-    search_fields = filter_fields
+    search_fields = filterset_fields
-    ordering = ('name',)
     serializer_class = serializers.AccountBackupSerializer
 
 

apps/accounts/api/automations/base.py

@@ -20,8 +20,8 @@ __all__ = [
 class AutomationAssetsListApi(generics.ListAPIView):
     model = BaseAutomation
     serializer_class = serializers.AutomationAssetsSerializer
-    filter_fields = ("name", "address")
+    filterset_fields = ("name", "address")
-    search_fields = filter_fields
+    search_fields = filterset_fields
 
     def get_object(self):
         pk = self.kwargs.get('pk')

apps/accounts/api/automations/change_secret.py

@@ -6,9 +6,12 @@ from rest_framework.response import Response
 
 from accounts import serializers
 from accounts.const import AutomationTypes
+from accounts.filters import ChangeSecretRecordFilterSet
 from accounts.models import ChangeSecretAutomation, ChangeSecretRecord
 from accounts.tasks import execute_automation_record_task
+from authentication.permissions import UserConfirmation, ConfirmType
 from orgs.mixins.api import OrgBulkModelViewSet, OrgGenericViewSet
+from rbac.permissions import RBACPermission
 from .base import (
     AutomationAssetsListApi, AutomationRemoveAssetApi, AutomationAddAssetApi,
     AutomationNodeAddRemoveApi, AutomationExecutionViewSet
@@ -24,35 +27,54 @@ __all__ = [
 
 class ChangeSecretAutomationViewSet(OrgBulkModelViewSet):
     model = ChangeSecretAutomation
-    filter_fields = ('name', 'secret_type', 'secret_strategy')
+    filterset_fields = ('name', 'secret_type', 'secret_strategy')
-    search_fields = filter_fields
+    search_fields = filterset_fields
     serializer_class = serializers.ChangeSecretAutomationSerializer
 
 
 class ChangeSecretRecordViewSet(mixins.ListModelMixin, OrgGenericViewSet):
-    serializer_class = serializers.ChangeSecretRecordSerializer
+    filterset_class = ChangeSecretRecordFilterSet
-    filterset_fields = ('asset_id', 'execution_id')
     search_fields = ('asset__address',)
     tp = AutomationTypes.change_secret
+    serializer_classes = {
+        'default': serializers.ChangeSecretRecordSerializer,
+        'secret': serializers.ChangeSecretRecordViewSecretSerializer,
+    }
     rbac_perms = {
         'execute': 'accounts.add_changesecretexecution',
+        'secret': 'accounts.view_changesecretrecord',
     }
 
+    def get_permissions(self):
+        if self.action == 'secret':
+            self.permission_classes = [
+                RBACPermission,
+                UserConfirmation.require(ConfirmType.MFA)
+            ]
+        return super().get_permissions()
+
     def get_queryset(self):
         return ChangeSecretRecord.objects.all()
 
     @action(methods=['post'], detail=False, url_path='execute')
     def execute(self, request, *args, **kwargs):
-        record_id = request.data.get('record_id')
+        record_ids = request.data.get('record_ids')
-        record = self.get_queryset().filter(pk=record_id)
+        records = self.get_queryset().filter(id__in=record_ids)
-        if not record:
+        execution_count = records.values_list('execution_id', flat=True).distinct().count()
+        if execution_count != 1:
             return Response(
-                {'detail': 'record not found'},
+                {'detail': 'Only one execution is allowed to execute'},
-                status=status.HTTP_404_NOT_FOUND
+                status=status.HTTP_400_BAD_REQUEST
             )
-        task = execute_automation_record_task.delay(record_id, self.tp)
+        task = execute_automation_record_task.delay(record_ids, self.tp)
         return Response({'task': task.id}, status=status.HTTP_200_OK)
 
+    @action(methods=['get'], detail=True, url_path='secret')
+    def secret(self, request, *args, **kwargs):
+        instance = self.get_object()
+        serializer = self.get_serializer(instance)
+        return Response(serializer.data)
+
 
 class ChangSecretExecutionViewSet(AutomationExecutionViewSet):
     rbac_perms = (

apps/accounts/api/automations/gather_accounts.py

@@ -20,8 +20,8 @@ __all__ = [
 
 class GatherAccountsAutomationViewSet(OrgBulkModelViewSet):
     model = GatherAccountsAutomation
-    filter_fields = ('name',)
+    filterset_fields = ('name',)
-    search_fields = filter_fields
+    search_fields = filterset_fields
     serializer_class = serializers.GatherAccountAutomationSerializer
 
 

apps/accounts/api/automations/push_account.py

@@ -20,8 +20,8 @@ __all__ = [
 
 class PushAccountAutomationViewSet(OrgBulkModelViewSet):
     model = PushAccountAutomation
-    filter_fields = ('name', 'secret_type', 'secret_strategy')
+    filterset_fields = ('name', 'secret_type', 'secret_strategy')
-    search_fields = filter_fields
+    search_fields = filterset_fields
     serializer_class = serializers.PushAccountAutomationSerializer
 
 

apps/accounts/automations/backup_account/handlers.py

@@ -3,13 +3,13 @@ import time
 from collections import defaultdict, OrderedDict
 
 from django.conf import settings
-from openpyxl import Workbook
 from rest_framework import serializers
+from xlsxwriter import Workbook
 
-from accounts.const.automation import AccountBackupType
+from accounts.const import AccountBackupType
+from accounts.models.automations.backup_account import AccountBackupAutomation
 from accounts.notifications import AccountBackupExecutionTaskMsg, AccountBackupByObjStorageExecutionTaskMsg
 from accounts.serializers import AccountSecretSerializer
-from accounts.models.automations.backup_account import AccountBackupAutomation
 from assets.const import AllTypes
 from common.utils.file import encrypt_and_compress_zip_file, zip_files
 from common.utils.timezone import local_now_filename, local_now_display
@@ -144,10 +144,11 @@ class AccountBackupHandler:
 
         wb = Workbook(filename)
         for sheet, data in data_map.items():
-            ws = wb.create_sheet(str(sheet))
+            ws = wb.add_worksheet(str(sheet))
-            for row in data:
+            for row_index, row_data in enumerate(data):
-                ws.append(row)
+                for col_index, col_data in enumerate(row_data):
-        wb.save(filename)
+                    ws.write_string(row_index, col_index, col_data)
+        wb.close()
         files.append(filename)
         timedelta = round((time.time() - time_start), 2)
         print('创建备份文件完成: 用时 {}s'.format(timedelta))
@@ -167,9 +168,8 @@ class AccountBackupHandler:
             if not user.secret_key:
                 attachment_list = []
             else:
-                password = user.secret_key.encode('utf8')
                 attachment = os.path.join(PATH, f'{plan_name}-{local_now_filename()}-{time.time()}.zip')
-                encrypt_and_compress_zip_file(attachment, password, files)
+                encrypt_and_compress_zip_file(attachment, user.secret_key, files)
                 attachment_list = [attachment, ]
             AccountBackupExecutionTaskMsg(plan_name, user).publish(attachment_list)
             print('邮件已发送至{}({})'.format(user, user.email))
@@ -190,7 +190,6 @@ class AccountBackupHandler:
             attachment = os.path.join(PATH, f'{plan_name}-{local_now_filename()}-{time.time()}.zip')
             if password:
                 print('\033[32m>>> 使用加密密码对文件进行加密中\033[0m')
-                password = password.encode('utf8')
                 encrypt_and_compress_zip_file(attachment, password, files)
             else:
                 zip_files(attachment, files)

apps/accounts/automations/change_secret/custom/ssh/main.yml

@@ -1,7 +1,6 @@
 - hosts: custom
   gather_facts: no
   vars:
-    asset_port: "{{ jms_asset.protocols | selectattr('name', 'equalto', 'ssh') | map(attribute='port') | first }}"
     ansible_connection: local
     ansible_become: false
 
@@ -9,7 +8,7 @@
     - name: Test privileged account (paramiko)
       ssh_ping:
         login_host: "{{ jms_asset.address }}"
-        login_port: "{{ asset_port }}"
+        login_port: "{{ jms_asset.port }}"
         login_user: "{{ jms_account.username }}"
         login_password: "{{ jms_account.secret }}"
         login_secret_type: "{{ jms_account.secret_type }}"
@@ -19,6 +18,8 @@
         become_user: "{{ custom_become_user | default('') }}"
         become_password: "{{ custom_become_password | default('') }}"
         become_private_key_path: "{{ custom_become_private_key_path | default(None) }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
+        gateway_args: "{{ jms_asset.ansible_ssh_common_args | default(None) }}"
       register: ping_info
       delegate_to: localhost
 
@@ -27,7 +28,7 @@
         login_user: "{{ jms_account.username }}"
         login_password: "{{ jms_account.secret }}"
         login_host: "{{ jms_asset.address }}"
-        login_port: "{{ asset_port }}"
+        login_port: "{{ jms_asset.port }}"
         login_secret_type: "{{ jms_account.secret_type }}"
         login_private_key_path: "{{ jms_account.private_key_path }}"
         become: "{{ custom_become | default(False) }}"
@@ -49,10 +50,12 @@
         login_user: "{{ account.username }}"
         login_password: "{{ account.secret }}"
         login_host: "{{ jms_asset.address }}"
-        login_port: "{{ asset_port }}"
+        login_port: "{{ jms_asset.port }}"
         become: "{{ account.become.ansible_become | default(False) }}"
         become_method: su
         become_user: "{{ account.become.ansible_user | default('') }}"
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
+        gateway_args: "{{ jms_asset.ansible_ssh_common_args | default(None) }}"
       delegate_to: localhost

apps/accounts/automations/change_secret/custom/ssh/manifest.yml

@@ -6,15 +6,27 @@ category:
 type:
   - all
 method: change_secret
+protocol: ssh
+priority: 50
 params:
   - name: commands
     type: list
-    label: '自定义命令'
+    label: "{{ 'Params commands label' | trans }}"
     default: [ '' ]
-    help_text: '自定义命令中如需包含账号的 账号、密码、SSH 连接的用户密码 字段，<br />请使用 &#123;username&#125;、&#123;password&#125;、&#123;login_password&#125;格式，执行任务时会进行替换 。<br />比如针对 Cisco 主机进行改密，一般需要配置五条命令：<br />1. enable<br />2. &#123;login_password&#125;<br />3. configure terminal<br />4. username &#123;username&#125; privilege 0 password &#123;password&#125; <br />5. end'
+    help_text: "{{ 'Params commands help text' | trans }}"
 
 i18n:
   SSH account change secret:
-    zh: 使用 SSH 命令行自定义改密
+    zh: '使用 SSH 命令行自定义改密'
-    ja: SSH コマンドライン方式でカスタムパスワード変更
+    ja: 'SSH コマンドライン方式でカスタムパスワード変更'
-    en: Custom password change by SSH command line
+    en: 'Custom password change by SSH command line'
+
+  Params commands help text:
+    zh: '自定义命令中如需包含账号的 账号、密码、SSH 连接的用户密码 字段，<br />请使用 &#123;username&#125;、&#123;password&#125;、&#123;login_password&#125;格式，执行任务时会进行替换 。<br />比如针对 Cisco 主机进行改密，一般需要配置五条命令：<br />1. enable<br />2. &#123;login_password&#125;<br />3. configure terminal<br />4. username &#123;username&#125; privilege 0 password &#123;password&#125; <br />5. end'
+    ja: 'カスタム コマンドに SSH 接続用のアカウント番号、パスワード、ユーザー パスワード フィールドを含める必要がある場合は、<br />&#123;ユーザー名&#125;、&#123;パスワード&#125;、&#123;login_password& を使用してください。 # 125; 形式。タスクの実行時に置き換えられます。 <br />たとえば、Cisco ホストのパスワードを変更するには、通常、次の 5 つのコマンドを設定する必要があります:<br />1.enable<br />2.&#123;login_password&#125;<br />3 .ターミナルの設定<br / >4. ユーザー名 &#123;ユーザー名&#125; 権限 0 パスワード &#123;パスワード&#125; <br />5. 終了'
+    en: 'If the custom command needs to include the account number, password, and user password field for SSH connection,<br />Please use &#123;username&#125;, &#123;password&#125;, &#123;login_password&# 125; format, which will be replaced when executing the task. <br />For example, to change the password of a Cisco host, you generally need to configure five commands:<br />1. enable<br />2. &#123;login_password&#125;<br />3. configure terminal<br / >4. username &#123;username&#125; privilege 0 password &#123;password&#125; <br />5. end'
+
+  Params commands label:
+    zh: '自定义命令'
+    ja: 'カスタムコマンド'
+    en: 'Custom command'

apps/accounts/automations/change_secret/database/mysql/main.yml

@@ -3,6 +3,7 @@
   vars:
     ansible_python_interpreter: /opt/py3/bin/python
     db_name: "{{ jms_asset.spec_info.db_name }}"
+    check_ssl: "{{ jms_asset.spec_info.use_ssl and not jms_asset.spec_info.allow_invalid_cert }}"
 
   tasks:
     - name: Test MySQL connection
@@ -11,10 +12,10 @@
         login_password: "{{ jms_account.secret }}"
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
-        check_hostname: "{{ omit if not jms_asset.spec_info.use_ssl else jms_asset.spec_info.allow_invalid_cert }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
-        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
-        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
-        client_key: "{{ jms_asset.secret_info.client_key  | default(omit) }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
         filter: version
       register: db_info
 
@@ -28,10 +29,10 @@
         login_password: "{{ jms_account.secret }}"
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
-        check_hostname: "{{ omit if not jms_asset.spec_info.use_ssl else jms_asset.spec_info.allow_invalid_cert }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
-        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
-        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
-        client_key: "{{ jms_asset.secret_info.client_key  | default(omit) }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
         name: "{{ account.username }}"
         password: "{{ account.secret }}"
         host: "%"
@@ -45,8 +46,8 @@
         login_password: "{{ account.secret }}"
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
-        check_hostname: "{{ omit if not jms_asset.spec_info.use_ssl else jms_asset.spec_info.allow_invalid_cert }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
-        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
-        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
-        client_key: "{{ jms_asset.secret_info.client_key  | default(omit) }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
         filter: version

apps/accounts/automations/change_secret/database/oracle/main.yml

@@ -39,3 +39,4 @@
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
         login_database: "{{ jms_asset.spec_info.db_name }}"
+        mode: "{{ account.mode }}"

apps/accounts/automations/change_secret/host/aix/main.yml

@@ -85,6 +85,7 @@
         become_user: "{{ account.become.ansible_user | default('') }}"
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
       when: account.secret_type == "password"
       delegate_to: localhost
 
@@ -95,5 +96,6 @@
         login_user: "{{ account.username }}"
         login_private_key_path: "{{ account.private_key_path  }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
       when: account.secret_type == "ssh_key"
       delegate_to: localhost

apps/accounts/automations/change_secret/host/posix/main.yml

@@ -85,6 +85,7 @@
         become_user: "{{ account.become.ansible_user | default('') }}"
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
       when: account.secret_type == "password"
       delegate_to: localhost
 
@@ -95,5 +96,6 @@
         login_user: "{{ account.username }}"
         login_private_key_path: "{{ account.private_key_path  }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
       when: account.secret_type == "ssh_key"
       delegate_to: localhost

apps/accounts/automations/change_secret/host/windows_rdp_verify/manifest.yml

@@ -5,6 +5,7 @@ method: change_secret
 category: host
 type:
   - windows
+priority: 49
 params:
   - name: groups
     type: str

apps/accounts/automations/change_secret/manager.py

@@ -4,11 +4,12 @@ from copy import deepcopy
 
 from django.conf import settings
 from django.utils import timezone
-from openpyxl import Workbook
+from django.utils.translation import gettext_lazy as _
+from xlsxwriter import Workbook
 
-from accounts.const import AutomationTypes, SecretType, SSHKeyStrategy, SecretStrategy
+from accounts.const import AutomationTypes, SecretType, SSHKeyStrategy, SecretStrategy, ChangeSecretRecordStatusChoice
 from accounts.models import ChangeSecretRecord
-from accounts.notifications import ChangeSecretExecutionTaskMsg
+from accounts.notifications import ChangeSecretExecutionTaskMsg, ChangeSecretFailedMsg
 from accounts.serializers import ChangeSecretRecordBackUpSerializer
 from assets.const import HostTypes
 from common.utils import get_logger
@@ -26,7 +27,7 @@ class ChangeSecretManager(AccountBasePlaybookManager):
 
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
-        self.record_id = self.execution.snapshot.get('record_id')
+        self.record_map = self.execution.snapshot.get('record_map', {})
         self.secret_type = self.execution.snapshot.get('secret_type')
         self.secret_strategy = self.execution.snapshot.get(
             'secret_strategy', SecretStrategy.custom
@@ -118,14 +119,24 @@ class ChangeSecretManager(AccountBasePlaybookManager):
             else:
                 new_secret = self.get_secret(secret_type)
 
-            if self.record_id is None:
+            if new_secret is None:
+                print(f'new_secret is None, account: {account}')
+                continue
+
+            asset_account_id = f'{asset.id}-{account.id}'
+            if asset_account_id not in self.record_map:
                 recorder = ChangeSecretRecord(
                     asset=asset, account=account, execution=self.execution,
                     old_secret=account.secret, new_secret=new_secret,
                 )
                 records.append(recorder)
             else:
-                recorder = ChangeSecretRecord.objects.get(id=self.record_id)
+                record_id = self.record_map[asset_account_id]
+                try:
+                    recorder = ChangeSecretRecord.objects.get(id=record_id)
+                except ChangeSecretRecord.DoesNotExist:
+                    print(f"Record {record_id} not found")
+                    continue
 
             self.name_recorder_mapper[h['name']] = recorder
 
@@ -139,7 +150,7 @@ class ChangeSecretManager(AccountBasePlaybookManager):
                 'name': account.name,
                 'username': account.username,
                 'secret_type': secret_type,
-                'secret': new_secret,
+                'secret': account.escape_jinja2_syntax(new_secret),
                 'private_key_path': private_key_path,
                 'become': account.get_ansible_become_auth(),
             }
@@ -153,24 +164,43 @@ class ChangeSecretManager(AccountBasePlaybookManager):
         recorder = self.name_recorder_mapper.get(host)
         if not recorder:
             return
-        recorder.status = 'success'
+        recorder.status = ChangeSecretRecordStatusChoice.success.value
         recorder.date_finished = timezone.now()
-        recorder.save()
+
         account = recorder.account
         if not account:
             print("Account not found, deleted ?")
             return
         account.secret = recorder.new_secret
-        account.save(update_fields=['secret'])
+        account.date_updated = timezone.now()
+
+        max_retries = 3
+        retry_count = 0
+
+        while retry_count < max_retries:
+            try:
+                recorder.save()
+                account.save(update_fields=['secret', 'version', 'date_updated'])
+                break
+            except Exception as e:
+                retry_count += 1
+                if retry_count == max_retries:
+                    self.on_host_error(host, str(e), result)
+                else:
+                    print(f'retry {retry_count} times for {host} recorder save error: {e}')
+                    time.sleep(1)
 
     def on_host_error(self, host, error, result):
         recorder = self.name_recorder_mapper.get(host)
         if not recorder:
             return
-        recorder.status = 'failed'
+        recorder.status = ChangeSecretRecordStatusChoice.failed.value
         recorder.date_finished = timezone.now()
         recorder.error = error
+        try:
             recorder.save()
+        except Exception as e:
+            print(f"\033[31m Save {host} recorder error: {e} \033[0m\n")
 
     def on_runner_failed(self, runner, e):
         logger.error("Account error: ", e)
@@ -182,23 +212,56 @@ class ChangeSecretManager(AccountBasePlaybookManager):
             return False
         return True
 
+    @staticmethod
+    def get_summary(recorders):
+        total, succeed, failed = 0, 0, 0
+        for recorder in recorders:
+            if recorder.status == ChangeSecretRecordStatusChoice.success.value:
+                succeed += 1
+            else:
+                failed += 1
+            total += 1
+
+        summary = _('Success: %s, Failed: %s, Total: %s') % (succeed, failed, total)
+        return summary
+
     def run(self, *args, **kwargs):
         if self.secret_type and not self.check_secret():
             return
         super().run(*args, **kwargs)
-        if self.record_id:
+        recorders = list(self.name_recorder_mapper.values())
-            return
+        summary = self.get_summary(recorders)
-        recorders = self.name_recorder_mapper.values()
+        print(summary, end='')
-        recorders = list(recorders)
+
-        self.send_recorder_mail(recorders)
+        if self.record_map:
+            return
+
+        failed_recorders = [
+            r for r in recorders
+            if r.status == ChangeSecretRecordStatusChoice.failed.value
+        ]
 
-    def send_recorder_mail(self, recorders):
         recipients = self.execution.recipients
-        if not recorders or not recipients:
+        recipients = User.objects.filter(id__in=list(recipients.keys()))
+        if not recipients:
             return
 
-        recipients = User.objects.filter(id__in=list(recipients.keys()))
+        if failed_recorders:
+            name = self.execution.snapshot.get('name')
+            execution_id = str(self.execution.id)
+            _ids = [r.id for r in failed_recorders]
+            asset_account_errors = ChangeSecretRecord.objects.filter(
+                id__in=_ids).values_list('asset__name', 'account__username', 'error')
 
+            for user in recipients:
+                ChangeSecretFailedMsg(name, execution_id, user, asset_account_errors).publish()
+
+        if not recorders:
+            return
+
+        self.send_recorder_mail(recipients, recorders, summary)
+
+    def send_recorder_mail(self, recipients, recorders, summary):
         name = self.execution.snapshot['name']
         path = os.path.join(os.path.dirname(settings.BASE_DIR), 'tmp')
         filename = os.path.join(path, f'{name}-{local_now_filename()}-{time.time()}.xlsx')
@@ -208,11 +271,10 @@ class ChangeSecretManager(AccountBasePlaybookManager):
         for user in recipients:
             attachments = []
             if user.secret_key:
-                password = user.secret_key.encode('utf8')
                 attachment = os.path.join(path, f'{name}-{local_now_filename()}-{time.time()}.zip')
-                encrypt_and_compress_zip_file(attachment, password, [filename])
+                encrypt_and_compress_zip_file(attachment, user.secret_key, [filename])
                 attachments = [attachment]
-            ChangeSecretExecutionTaskMsg(name, user).publish(attachments)
+            ChangeSecretExecutionTaskMsg(name, user, summary).publish(attachments)
         os.remove(filename)
 
     @staticmethod
@@ -227,8 +289,9 @@ class ChangeSecretManager(AccountBasePlaybookManager):
 
         rows.insert(0, header)
         wb = Workbook(filename)
-        ws = wb.create_sheet('Sheet1')
+        ws = wb.add_worksheet('Sheet1')
-        for row in rows:
+        for row_index, row_data in enumerate(rows):
-            ws.append(row)
+            for col_index, col_data in enumerate(row_data):
-        wb.save(filename)
+                ws.write_string(row_index, col_index, col_data)
+        wb.close()
         return True

apps/accounts/automations/endpoint.py

@@ -1,8 +1,9 @@
-from .push_account.manager import PushAccountManager
-from .change_secret.manager import ChangeSecretManager
-from .verify_account.manager import VerifyAccountManager
 from .backup_account.manager import AccountBackupManager
+from .change_secret.manager import ChangeSecretManager
 from .gather_accounts.manager import GatherAccountsManager
+from .push_account.manager import PushAccountManager
+from .remove_account.manager import RemoveAccountManager
+from .verify_account.manager import VerifyAccountManager
 from .verify_gateway_account.manager import VerifyGatewayAccountManager
 from ..const import AutomationTypes
 
@@ -12,6 +13,7 @@ class ExecutionManager:
         AutomationTypes.push_account: PushAccountManager,
         AutomationTypes.change_secret: ChangeSecretManager,
         AutomationTypes.verify_account: VerifyAccountManager,
+        AutomationTypes.remove_account: RemoveAccountManager,
         AutomationTypes.gather_accounts: GatherAccountsManager,
         AutomationTypes.verify_gateway_account: VerifyGatewayAccountManager,
         # TODO 后期迁移到自动化策略中

apps/accounts/automations/gather_accounts/database/mysql/main.yml

@@ -2,6 +2,7 @@
   gather_facts: no
   vars:
     ansible_python_interpreter: /opt/py3/bin/python
+    check_ssl: "{{ jms_asset.spec_info.use_ssl and not jms_asset.spec_info.allow_invalid_cert }}"
 
   tasks:
     - name: Get info
@@ -10,10 +11,10 @@
         login_password: "{{ jms_account.secret }}"
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
-        check_hostname: "{{ omit if not jms_asset.spec_info.use_ssl else jms_asset.spec_info.allow_invalid_cert }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
-        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
-        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
-        client_key: "{{ jms_asset.secret_info.client_key  | default(omit) }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
         filter: users
       register: db_info
 

apps/accounts/automations/gather_accounts/host/windows/main.yml

@@ -1,9 +1,10 @@
 - hosts: demo
   gather_facts: no
   tasks:
-    - name: Gather posix account
+    - name: Gather windows account
       ansible.builtin.win_shell: net user
       register: result
+      ignore_errors: true
 
     - name: Define info by set_fact
       ansible.builtin.set_fact:

apps/accounts/automations/gather_accounts/manager.py

@@ -51,14 +51,22 @@ class GatherAccountsManager(AccountBasePlaybookManager):
         data = self.generate_data(asset, result)
         self.asset_account_info[asset] = data
 
+    @staticmethod
+    def get_nested_info(data, *keys):
+        for key in keys:
+            data = data.get(key, {})
+            if not data:
+                break
+        return data
+
     def on_host_success(self, host, result):
-        info = result.get('debug', {}).get('res', {}).get('info', {})
+        info = self.get_nested_info(result, 'debug', 'res', 'info')
         asset = self.host_asset_mapper.get(host)
         if asset and info:
             result = self.filter_success_result(asset.type, info)
             self.collect_asset_account_info(asset, result)
         else:
-            logger.error(f'Not found {host} info')
+            print(f'\033[31m Not found {host} info \033[0m\n')
 
     def update_or_create_accounts(self):
         for asset, data in self.asset_account_info.items():
@@ -72,7 +80,7 @@ class GatherAccountsManager(AccountBasePlaybookManager):
                     )
                     gathered_accounts.append(gathered_account)
                 if not self.is_sync_account:
-                    return
+                    continue
                 GatheredAccount.sync_accounts(gathered_accounts)
 
     def run(self, *args, **kwargs):

apps/accounts/automations/push_account/database/mysql/main.yml

@@ -3,6 +3,7 @@
   vars:
     ansible_python_interpreter: /opt/py3/bin/python
     db_name: "{{ jms_asset.spec_info.db_name }}"
+    check_ssl: "{{ jms_asset.spec_info.use_ssl and not jms_asset.spec_info.allow_invalid_cert }}"
 
   tasks:
     - name: Test MySQL connection
@@ -11,10 +12,10 @@
         login_password: "{{ jms_account.secret }}"
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
-        check_hostname: "{{ omit if not jms_asset.spec_info.use_ssl else jms_asset.spec_info.allow_invalid_cert }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
-        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
-        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
-        client_key: "{{ jms_asset.secret_info.client_key  | default(omit) }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
         filter: version
       register: db_info
 
@@ -28,10 +29,10 @@
         login_password: "{{ jms_account.secret }}"
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
-        check_hostname: "{{ omit if not jms_asset.spec_info.use_ssl else jms_asset.spec_info.allow_invalid_cert }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
-        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
-        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
-        client_key: "{{ jms_asset.secret_info.client_key  | default(omit) }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
         name: "{{ account.username }}"
         password: "{{ account.secret }}"
         host: "%"
@@ -45,8 +46,8 @@
         login_password: "{{ account.secret }}"
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
-        check_hostname: "{{ omit if not jms_asset.spec_info.use_ssl else jms_asset.spec_info.allow_invalid_cert }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
-        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
-        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
-        client_key: "{{ jms_asset.secret_info.client_key  | default(omit) }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
         filter: version

apps/accounts/automations/push_account/database/oracle/main.yml

@@ -39,3 +39,4 @@
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
         login_database: "{{ jms_asset.spec_info.db_name }}"
+        mode: "{{ account.mode }}"

apps/accounts/automations/push_account/host/aix/main.yml

@@ -85,6 +85,7 @@
         become_user: "{{ account.become.ansible_user | default('') }}"
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
       when: account.secret_type == "password"
       delegate_to: localhost
 
@@ -95,6 +96,7 @@
         login_user: "{{ account.username }}"
         login_private_key_path: "{{ account.private_key_path  }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
       when: account.secret_type == "ssh_key"
       delegate_to: localhost
 

apps/accounts/automations/push_account/host/aix/manifest.yml

@@ -9,7 +9,7 @@ params:
     type: str
     label: 'Sudo'
     default: '/bin/whoami'
-    help_text: '使用逗号分隔多个命令，如: /bin/whoami,/sbin/ifconfig'
+    help_text: "{{ 'Params sudo help text' | trans }}"
 
   - name: shell
     type: str
@@ -18,19 +18,44 @@ params:
 
   - name: home
     type: str
-    label: '家目录'
+    label: "{{ 'Params home label' | trans }}"
     default: ''
-    help_text: '默认家目录 /home/系统用户名: /home/username'
+    help_text: "{{ 'Params home help text' | trans }}"
 
   - name: groups
     type: str
-    label: '用户组'
+    label: "{{ 'Params groups label' | trans }}"
     default: ''
-    help_text: '请输入用户组，多个用户组使用逗号分隔（需填写已存在的用户组）'
+    help_text: "{{ 'Params groups help text' | trans }}"
 
 i18n:
   Aix account push:
-    zh: 使用 Ansible 模块 user 执行 Aix 账号推送 (DES)
+    zh: '使用 Ansible 模块 user 执行 Aix 账号推送 (DES)'
-    ja: Ansible user モジュールを使用して Aix アカウントをプッシュする (DES)
+    ja: 'Ansible user モジュールを使用して Aix アカウントをプッシュする (DES)'
-    en: Using Ansible module user to push account (DES)
+    en: 'Using Ansible module user to push account (DES)'
+
+  Params sudo help text:
+    zh: '使用逗号分隔多个命令，如: /bin/whoami,/sbin/ifconfig'
+    ja: 'コンマで区切って複数のコマンドを入力してください。例: /bin/whoami,/sbin/ifconfig'
+    en: 'Use commas to separate multiple commands, such as: /bin/whoami,/sbin/ifconfig'
+
+  Params home help text:
+    zh: '默认家目录 /home/{账号用户名}'
+    ja: 'デフォルトのホームディレクトリ /home/{アカウントユーザ名}'
+    en: 'Default home directory /home/{account username}'
+
+  Params groups help text:
+    zh: '请输入用户组，多个用户组使用逗号分隔（需填写已存在的用户组）'
+    ja: 'グループを入力してください。複数のグループはコンマで区切ってください（既存のグループを入力してください）'
+    en: 'Please enter the group. Multiple groups are separated by commas (please enter the existing group)'
+
+  Params home label:
+    zh: '家目录'
+    ja: 'ホームディレクトリ'
+    en: 'Home'
+
+  Params groups label:
+    zh: '用户组'
+    ja: 'グループ'
+    en: 'Groups'
 

apps/accounts/automations/push_account/host/posix/main.yml

@@ -85,6 +85,7 @@
         become_user: "{{ account.become.ansible_user | default('') }}"
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
       when: account.secret_type == "password"
       delegate_to: localhost
 
@@ -95,6 +96,7 @@
         login_user: "{{ account.username }}"
         login_private_key_path: "{{ account.private_key_path  }}"
         gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
       when: account.secret_type == "ssh_key"
       delegate_to: localhost
 

apps/accounts/automations/push_account/host/posix/manifest.yml

@@ -10,7 +10,7 @@ params:
     type: str
     label: 'Sudo'
     default: '/bin/whoami'
-    help_text: '使用逗号分隔多个命令，如: /bin/whoami,/sbin/ifconfig'
+    help_text: "{{ 'Params sudo help text' | trans }}"
 
   - name: shell
     type: str
@@ -20,18 +20,43 @@ params:
 
   - name: home
     type: str
-    label: '家目录'
+    label: "{{ 'Params home label' | trans }}"
     default: ''
-    help_text: '默认家目录 /home/系统用户名: /home/username'
+    help_text: "{{ 'Params home help text' | trans }}"
 
   - name: groups
     type: str
-    label: '用户组'
+    label: "{{ 'Params groups label' | trans }}"
     default: ''
-    help_text: '请输入用户组，多个用户组使用逗号分隔（需填写已存在的用户组）'
+    help_text: "{{ 'Params groups help text' | trans }}"
 
 i18n:
   Posix account push:
-    zh: 使用 Ansible 模块 user 执行账号推送 (sha512)
+    zh: '使用 Ansible 模块 user 执行账号推送 (sha512)'
-    ja: Ansible user モジュールを使用してアカウントをプッシュする (sha512)
+    ja: 'Ansible user モジュールを使用してアカウントをプッシュする (sha512)'
-    en: Using Ansible module user to push account (sha512)
+    en: 'Using Ansible module user to push account (sha512)'
+
+  Params sudo help text:
+    zh: '使用逗号分隔多个命令，如: /bin/whoami,/sbin/ifconfig'
+    ja: 'コンマで区切って複数のコマンドを入力してください。例: /bin/whoami,/sbin/ifconfig'
+    en: 'Use commas to separate multiple commands, such as: /bin/whoami,/sbin/ifconfig'
+
+  Params home help text:
+    zh: '默认家目录 /home/{账号用户名}'
+    ja: 'デフォルトのホームディレクトリ /home/{アカウントユーザ名}'
+    en: 'Default home directory /home/{account username}'
+
+  Params groups help text:
+    zh: '请输入用户组，多个用户组使用逗号分隔（需填写已存在的用户组）'
+    ja: 'グループを入力してください。複数のグループはコンマで区切ってください（既存のグループを入力してください）'
+    en: 'Please enter the group. Multiple groups are separated by commas (please enter the existing group)'
+
+  Params home label:
+    zh: '家目录'
+    ja: 'ホームディレクトリ'
+    en: 'Home'
+
+  Params groups label:
+    zh: '用户组'
+    ja: 'グループ'
+    en: 'Groups'

apps/accounts/automations/push_account/host/windows/manifest.yml

@@ -10,10 +10,15 @@ params:
     type: str
     label: '用户组'
     default: 'Users,Remote Desktop Users'
-    help_text: '请输入用户组，多个用户组使用逗号分隔（需填写已存在的用户组）'
+    help_text: "{{ 'Params groups help text' | trans }}"
 
 i18n:
   Windows account push:
-    zh: 使用 Ansible 模块 win_user 执行 Windows 账号推送
+    zh: '使用 Ansible 模块 win_user 执行 Windows 账号推送'
-    ja: Ansible win_user モジュールを使用して Windows アカウントをプッシュする
+    ja: 'Ansible win_user モジュールを使用して Windows アカウントをプッシュする'
-    en: Using Ansible module win_user to push account
+    en: 'Using Ansible module win_user to push account'
+
+  Params groups help text:
+    zh: '请输入用户组，多个用户组使用逗号分隔（需填写已存在的用户组）'
+    ja: 'グループを入力してください。複数のグループはコンマで区切ってください（既存のグループを入力してください）'
+    en: 'Please enter the group. Multiple groups are separated by commas (please enter the existing group)'

apps/accounts/automations/push_account/host/windows_rdp_verify/manifest.yml

@@ -5,15 +5,21 @@ method: push_account
 category: host
 type:
   - windows
+priority: 49
 params:
   - name: groups
     type: str
     label: '用户组'
     default: 'Users,Remote Desktop Users'
-    help_text: '请输入用户组，多个用户组使用逗号分隔（需填写已存在的用户组）'
+    help_text: "{{ 'Params groups help text' | trans }}"
 
 i18n:
   Windows account push rdp verify:
-    zh: 使用 Ansible 模块 win_user 执行 Windows 账号推送 RDP 协议测试最后的可连接性
+    zh: '使用 Ansible 模块 win_user 执行 Windows 账号推送（最后使用 Python 模块 pyfreerdp 验证账号的可连接性）'
-    ja: Ansibleモジュールwin_userがWindowsアカウントプッシュRDPプロトコルテストを実行する最後の接続性
+    ja: 'Ansible モジュール win_user を使用して Windows アカウントのプッシュを実行します (最後に Python モジュール pyfreerdp を使用してアカウントの接続性を確認します)'
-    en: Using the Ansible module win_user performs Windows account push RDP protocol testing for final connectivity
+    en: 'Use the Ansible module win_user to perform Windows account push (finally use the Python module pyfreerdp to verify the connectability of the account)'
+
+  Params groups help text:
+    zh: '请输入用户组，多个用户组使用逗号分隔（需填写已存在的用户组）'
+    ja: 'グループを入力してください。複数のグループはコンマで区切ってください（既存のグループを入力してください）'
+    en: 'Please enter the group. Multiple groups are separated by commas (please enter the existing group)'

apps/accounts/automations/remove_account/database/mongodb/main.yml

@@ -0,0 +1,21 @@
+- hosts: mongodb
+  gather_facts: no
+  vars:
+    ansible_python_interpreter: /opt/py3/bin/python
+
+  tasks:
+    - name: "Remove account"
+      mongodb_user:
+        login_user: "{{ jms_account.username }}"
+        login_password: "{{ jms_account.secret }}"
+        login_host: "{{ jms_asset.address }}"
+        login_port: "{{ jms_asset.port }}"
+        login_database: "{{ jms_asset.spec_info.db_name }}"
+        ssl: "{{ jms_asset.spec_info.use_ssl }}"
+        ssl_ca_certs: "{{ jms_asset.secret_info.ca_cert | default('') }}"
+        ssl_certfile: "{{ jms_asset.secret_info.client_key | default('') }}"
+        connection_options:
+          - tlsAllowInvalidHostnames: "{{ jms_asset.spec_info.allow_invalid_cert}}"
+        db: "{{ jms_asset.spec_info.db_name }}"
+        name: "{{ account.username }}"
+        state: absent

apps/accounts/automations/remove_account/database/mongodb/manifest.yml

@@ -0,0 +1,12 @@
+id: remove_account_mongodb
+name: "{{ 'MongoDB account remove' | trans }}"
+category: database
+type:
+  - mongodb
+method: remove_account
+
+i18n:
+  MongoDB account remove:
+    zh: 使用 Ansible 模块 mongodb 删除账号
+    ja: Ansible モジュール mongodb を使用してアカウントを削除する
+    en: Delete account using Ansible module mongodb

apps/accounts/automations/remove_account/database/mysql/main.yml

@@ -0,0 +1,18 @@
+- hosts: mysql
+  gather_facts: no
+  vars:
+    ansible_python_interpreter: /opt/py3/bin/python
+
+  tasks:
+    - name: "Remove account"
+      community.mysql.mysql_user:
+        login_user: "{{ jms_account.username }}"
+        login_password: "{{ jms_account.secret }}"
+        login_host: "{{ jms_asset.address }}"
+        login_port: "{{ jms_asset.port }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
+        name: "{{ account.username }}"
+        state: absent

apps/accounts/automations/remove_account/database/mysql/manifest.yml

@@ -0,0 +1,14 @@
+id: remove_account_mysql
+name: "{{ 'MySQL account remove' | trans }}"
+category: database
+type:
+  - mysql
+  - mariadb
+method: remove_account
+
+i18n:
+  MySQL account remove:
+    zh: 使用 Ansible 模块 mysql_user 删除账号
+    ja: Ansible モジュール mysql_user を使用してアカウントを削除します
+    en: Use the Ansible module mysql_user to delete the account
+

apps/accounts/automations/remove_account/database/oracle/main.yml

@@ -0,0 +1,16 @@
+- hosts: oracle
+  gather_facts: no
+  vars:
+    ansible_python_interpreter: /opt/py3/bin/python
+
+  tasks:
+    - name: "Remove account"
+      oracle_user:
+        login_user: "{{ jms_account.username }}"
+        login_password: "{{ jms_account.secret }}"
+        login_host: "{{ jms_asset.address }}"
+        login_port: "{{ jms_asset.port }}"
+        login_database: "{{ jms_asset.spec_info.db_name }}"
+        mode: "{{ jms_account.mode }}"
+        name: "{{ account.username }}"
+        state: absent

apps/accounts/automations/remove_account/database/oracle/manifest.yml

@@ -0,0 +1,12 @@
+id: remove_account_oracle
+name: "{{ 'Oracle account remove' | trans }}"
+category: database
+type:
+  - oracle
+method: remove_account
+
+i18n:
+  Oracle account remove:
+    zh: 使用 Python 模块 oracledb 删除账号
+    ja: Python モジュール oracledb を使用してアカウントを検証する
+    en: Using Python module oracledb to verify account

apps/accounts/automations/remove_account/database/postgresql/main.yml

@@ -0,0 +1,15 @@
+- hosts: postgresql
+  gather_facts: no
+  vars:
+    ansible_python_interpreter: /opt/py3/bin/python
+
+  tasks:
+    - name: "Remove account"
+      community.postgresql.postgresql_user:
+        login_user: "{{ jms_account.username }}"
+        login_password: "{{ jms_account.secret }}"
+        login_host: "{{ jms_asset.address }}"
+        login_port: "{{ jms_asset.port }}"
+        db: "{{ jms_asset.spec_info.db_name }}"
+        name: "{{ account.username }}"
+        state: absent

apps/accounts/automations/remove_account/database/postgresql/manifest.yml

@@ -0,0 +1,12 @@
+id: remove_account_postgresql
+name: "{{ 'PostgreSQL account remove' | trans }}"
+category: database
+type:
+  - postgresql
+method: remove_account
+
+i18n:
+  PostgreSQL account remove:
+    zh: 使用 Ansible 模块 postgresql_user 删除账号
+    ja: Ansible モジュール postgresql_user を使用してアカウントを削除します
+    en: Use the Ansible module postgresql_user to delete the account

apps/accounts/automations/remove_account/database/sqlserver/main.yml

@@ -0,0 +1,14 @@
+- hosts: sqlserver
+  gather_facts: no
+  vars:
+    ansible_python_interpreter: /opt/py3/bin/python
+
+  tasks:
+    - name: "Remove account"
+      community.general.mssql_script:
+        login_user: "{{ jms_account.username }}"
+        login_password: "{{ jms_account.secret }}"
+        login_host: "{{ jms_asset.address }}"
+        login_port: "{{ jms_asset.port }}"
+        name: "{{ jms_asset.spec_info.db_name }}"
+        script: "DROP USER {{ account.username }}"

apps/accounts/automations/remove_account/database/sqlserver/manifest.yml

@@ -0,0 +1,12 @@
+id: remove_account_sqlserver
+name: "{{ 'SQLServer account remove' | trans }}"
+category: database
+type:
+  - sqlserver
+method: remove_account
+
+i18n:
+  SQLServer account remove:
+    zh: 使用 Ansible 模块 mssql 删除账号
+    ja: Ansible モジュール mssql を使用してアカウントを削除する
+    en: Use Ansible module mssql to delete account

apps/accounts/automations/remove_account/host/posix/main.yml

@@ -0,0 +1,26 @@
+- hosts: demo
+  gather_facts: no
+  tasks:
+    - name: "Get user home directory path"
+      ansible.builtin.shell:
+        cmd: "getent passwd {{ account.username }} | cut -d: -f6"
+      register: user_home_dir
+      ignore_errors: yes
+
+    - name: "Check if user home directory exists"
+      ansible.builtin.stat:
+        path: "{{ user_home_dir.stdout }}"
+      register: home_dir
+      when: user_home_dir.stdout != ""
+
+    - name: "Rename user home directory if it exists"
+      ansible.builtin.command:
+        cmd: "mv {{ user_home_dir.stdout }} {{ user_home_dir.stdout }}.bak"
+      when: home_dir.stat | default(false) and user_home_dir.stdout != ""
+
+    - name: "Remove account"
+      ansible.builtin.user:
+        name: "{{ account.username }}"
+        state: absent
+        remove: "{{ home_dir.stat.exists }}"
+      when: home_dir.stat | default(false)

apps/accounts/automations/remove_account/host/posix/manifest.yml

@@ -0,0 +1,13 @@
+id: remove_account_posix
+name: "{{ 'Posix account remove' | trans }}"
+category: host
+type:
+  - linux
+  - unix
+method: remove_account
+
+i18n:
+  Posix account remove:
+    zh: 使用 Ansible 模块 user 删除账号
+    ja: Ansible モジュール ユーザーを使用してアカウントを削除します
+    en: Use the Ansible module user to delete the account

apps/accounts/automations/remove_account/host/windows/main.yml

@@ -0,0 +1,9 @@
+- hosts: windows
+  gather_facts: no
+  tasks:
+    - name: "Remove account"
+      ansible.windows.win_user:
+        name: "{{ account.username }}"
+        state: absent
+        purge: yes
+        force: yes

apps/accounts/automations/remove_account/host/windows/manifest.yml

@@ -0,0 +1,13 @@
+id: remove_account_windows
+name: "{{ 'Windows account remove' | trans }}"
+version: 1
+method: remove_account
+category: host
+type:
+  - windows
+
+i18n:
+  Windows account remove:
+    zh: 使用 Ansible 模块 win_user 删除账号
+    ja: Ansible モジュール win_user を使用してアカウントを削除する
+    en: Use the Ansible module win_user to delete an account

apps/accounts/automations/remove_account/manager.py

@@ -0,0 +1,70 @@
+import os
+from copy import deepcopy
+
+from django.db.models import QuerySet
+
+from accounts.const import AutomationTypes
+from accounts.models import Account
+from common.utils import get_logger
+from ..base.manager import AccountBasePlaybookManager
+
+logger = get_logger(__name__)
+
+
+class RemoveAccountManager(AccountBasePlaybookManager):
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.host_account_mapper = {}
+
+    def prepare_runtime_dir(self):
+        path = super().prepare_runtime_dir()
+        ansible_config_path = os.path.join(path, 'ansible.cfg')
+
+        with open(ansible_config_path, 'w') as f:
+            f.write('[ssh_connection]\n')
+            f.write('ssh_args = -o ControlMaster=no -o ControlPersist=no\n')
+        return path
+
+    @classmethod
+    def method_type(cls):
+        return AutomationTypes.remove_account
+
+    def get_gather_accounts(self, privilege_account, gather_accounts: QuerySet):
+        gather_account_ids = self.execution.snapshot['gather_accounts']
+        gather_accounts = gather_accounts.filter(id__in=gather_account_ids)
+        gather_accounts = gather_accounts.exclude(
+            username__in=[privilege_account.username, 'root', 'Administrator']
+        )
+        return gather_accounts
+
+    def host_callback(self, host, asset=None, account=None, automation=None, path_dir=None, **kwargs):
+        if host.get('error'):
+            return host
+
+        gather_accounts = asset.gatheredaccount_set.all()
+        gather_accounts = self.get_gather_accounts(account, gather_accounts)
+
+        inventory_hosts = []
+
+        for gather_account in gather_accounts:
+            h = deepcopy(host)
+            h['name'] += '(' + gather_account.username + ')'
+            self.host_account_mapper[h['name']] = (asset, gather_account)
+            h['account'] = {'username': gather_account.username}
+            inventory_hosts.append(h)
+        return inventory_hosts
+
+    def on_host_success(self, host, result):
+        tuple_asset_gather_account = self.host_account_mapper.get(host)
+        if not tuple_asset_gather_account:
+            return
+        asset, gather_account = tuple_asset_gather_account
+        try:
+            Account.objects.filter(
+                asset_id=asset.id,
+                username=gather_account.username
+            ).delete()
+            gather_account.delete()
+        except Exception as e:
+            print(f'\033[31m Delete account {gather_account.username} failed: {e} \033[0m\n')

apps/accounts/automations/verify_account/custom/rdp/main.yml

@@ -3,12 +3,13 @@
   vars:
     ansible_shell_type: sh
     ansible_connection: local
+    ansible_python_interpreter: /opt/py3/bin/python
 
   tasks:
     - name: Verify account (pyfreerdp)
       rdp_ping:
         login_host: "{{ jms_asset.address }}"
-        login_port: "{{ jms_asset.protocols | selectattr('name', 'equalto', 'rdp') | map(attribute='port') | first }}"
+        login_port: "{{ jms_asset.port }}"
         login_user: "{{ account.username }}"
         login_password: "{{ account.secret }}"
         login_secret_type: "{{ account.secret_type }}"

apps/accounts/automations/verify_account/custom/rdp/manifest.yml

@@ -5,9 +5,11 @@ category:
 type:
   - windows
 method: verify_account
+protocol: rdp
+priority: 1
 
 i18n:
   Windows rdp account verify:
-    zh: 使用 Python 模块 pyfreerdp 验证账号
+    zh: '使用 Python 模块 pyfreerdp 验证账号'
-    ja: Python モジュール pyfreerdp を使用してアカウントを検証する
+    ja: 'Python モジュール pyfreerdp を使用してアカウントを検証する'
-    en: Using Python module pyfreerdp to verify account
+    en: 'Using Python module pyfreerdp to verify account'

apps/accounts/automations/verify_account/custom/ssh/main.yml

@@ -9,7 +9,7 @@
     - name: Verify account (paramiko)
       ssh_ping:
         login_host: "{{ jms_asset.address }}"
-        login_port: "{{ jms_asset.protocols | selectattr('name', 'equalto', 'ssh') | map(attribute='port') | first }}"
+        login_port: "{{ jms_asset.port }}"
         login_user: "{{ account.username }}"
         login_password: "{{ account.secret }}"
         login_secret_type: "{{ account.secret_type }}"
@@ -19,3 +19,5 @@
         become_user: "{{ account.become.ansible_user | default('') }}"
         become_password: "{{ account.become.ansible_password | default('') }}"
         become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
+        old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
+        gateway_args: "{{ jms_asset.ansible_ssh_common_args | default(None) }}"

apps/accounts/automations/verify_account/custom/ssh/manifest.yml

@@ -6,9 +6,11 @@ category:
 type:
   - all
 method: verify_account
+protocol: ssh
+priority: 50
 
 i18n:
   SSH account verify:
-    zh: 使用 Python 模块 paramiko 验证账号
+    zh: '使用 Python 模块 paramiko 验证账号'
-    ja: Python モジュール paramiko を使用してアカウントを検証する
+    ja: 'Python モジュール paramiko を使用してアカウントを検証する'
-    en: Using Python module paramiko to verify account
+    en: 'Using Python module paramiko to verify account'

apps/accounts/automations/verify_account/database/mongodb/main.yml

@@ -1,4 +1,4 @@
-- hosts: mongdb
+- hosts: mongodb
   gather_facts: no
   vars:
     ansible_python_interpreter: /opt/py3/bin/python

apps/accounts/automations/verify_account/database/mysql/main.yml

@@ -2,6 +2,7 @@
   gather_facts: no
   vars:
     ansible_python_interpreter: /opt/py3/bin/python
+    check_ssl: "{{ jms_asset.spec_info.use_ssl and not jms_asset.spec_info.allow_invalid_cert }}"
 
   tasks:
     - name: Verify account
@@ -10,8 +11,8 @@
         login_password: "{{ account.secret }}"
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
-        check_hostname: "{{ omit if not jms_asset.spec_info.use_ssl else jms_asset.spec_info.allow_invalid_cert }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
-        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
-        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
-        client_key: "{{ jms_asset.secret_info.client_key  | default(omit) }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
         filter: version

apps/accounts/automations/verify_account/manager.py

@@ -51,6 +51,9 @@ class VerifyAccountManager(AccountBasePlaybookManager):
             h['name'] += '(' + account.username + ')'
             self.host_account_mapper[h['name']] = account
             secret = account.secret
+            if secret is None:
+                print(f'account {account.name} secret is None')
+                continue
 
             private_key_path = None
             if account.secret_type == SecretType.SSH_KEY:
@@ -62,7 +65,7 @@ class VerifyAccountManager(AccountBasePlaybookManager):
                 'name': account.name,
                 'username': account.username,
                 'secret_type': account.secret_type,
-                'secret': secret,
+                'secret': account.escape_jinja2_syntax(secret),
                 'private_key_path': private_key_path,
                 'become': account.get_ansible_become_auth(),
             }
@@ -73,8 +76,14 @@ class VerifyAccountManager(AccountBasePlaybookManager):
 
     def on_host_success(self, host, result):
         account = self.host_account_mapper.get(host)
+        try:
             account.set_connectivity(Connectivity.OK)
+        except Exception as e:
+            print(f'\033[31m Update account {account.name} connectivity failed: {e} \033[0m\n')
 
     def on_host_error(self, host, error, result):
         account = self.host_account_mapper.get(host)
+        try:
             account.set_connectivity(Connectivity.ERR)
+        except Exception as e:
+            print(f'\033[31m Update account {account.name} connectivity failed: {e} \033[0m\n')

apps/accounts/const/account.py

@@ -15,6 +15,7 @@ class AliasAccount(TextChoices):
     INPUT = '@INPUT', _('Manual input')
     USER = '@USER', _('Dynamic user')
     ANON = '@ANON', _('Anonymous account')
+    SPEC = '@SPEC', _('Specified account')
 
     @classmethod
     def virtual_choices(cls):

apps/accounts/const/automation.py

@@ -16,7 +16,7 @@ DEFAULT_PASSWORD_RULES = {
 __all__ = [
     'AutomationTypes', 'SecretStrategy', 'SSHKeyStrategy', 'Connectivity',
     'DEFAULT_PASSWORD_LENGTH', 'DEFAULT_PASSWORD_RULES', 'TriggerChoice',
-    'PushAccountActionChoice', 'AccountBackupType'
+    'PushAccountActionChoice', 'AccountBackupType', 'ChangeSecretRecordStatusChoice',
 ]
 
 
@@ -24,6 +24,7 @@ class AutomationTypes(models.TextChoices):
     push_account = 'push_account', _('Push account')
     change_secret = 'change_secret', _('Change secret')
     verify_account = 'verify_account', _('Verify account')
+    remove_account = 'remove_account', _('Remove account')
     gather_accounts = 'gather_accounts', _('Gather accounts')
     verify_gateway_account = 'verify_gateway_account', _('Verify gateway account')
 
@@ -102,3 +103,9 @@ class AccountBackupType(models.TextChoices):
     email = 'email', _('Email')
     # 目前只支持sftp方式
     object_storage = 'object_storage', _('SFTP')
+
+
+class ChangeSecretRecordStatusChoice(models.TextChoices):
+    failed = 'failed', _('Failed')
+    success = 'success', _('Success')
+    pending = 'pending', _('Pending')

apps/accounts/filters.py

@@ -5,7 +5,7 @@ from django_filters import rest_framework as drf_filters
 
 from assets.models import Node
 from common.drf.filters import BaseFilterSet
-from .models import Account, GatheredAccount
+from .models import Account, GatheredAccount, ChangeSecretRecord
 
 
 class AccountFilterSet(BaseFilterSet):
@@ -51,6 +51,8 @@ class AccountFilterSet(BaseFilterSet):
 
 class GatheredAccountFilterSet(BaseFilterSet):
     node_id = drf_filters.CharFilter(method='filter_nodes')
+    asset_id = drf_filters.CharFilter(field_name='asset_id', lookup_expr='exact')
+    asset_name = drf_filters.CharFilter(field_name='asset__name', lookup_expr='icontains')
 
     @staticmethod
     def filter_nodes(queryset, name, value):
@@ -58,4 +60,14 @@ class GatheredAccountFilterSet(BaseFilterSet):
 
     class Meta:
         model = GatheredAccount
-        fields = ['id', 'asset_id', 'username']
+        fields = ['id', 'username']
+
+
+class ChangeSecretRecordFilterSet(BaseFilterSet):
+    asset_name = drf_filters.CharFilter(field_name='asset__name', lookup_expr='icontains')
+    account_username = drf_filters.CharFilter(field_name='account__username', lookup_expr='icontains')
+    execution_id = drf_filters.CharFilter(field_name='execution_id', lookup_expr='exact')
+
+    class Meta:
+        model = ChangeSecretRecord
+        fields = ['id', 'status', 'asset_id', 'execution']

apps/accounts/migrations/0007_alter_account_options.py

@@ -4,7 +4,6 @@ from django.db import migrations
 
 
 class Migration(migrations.Migration):
-
     dependencies = [
         ('accounts', '0006_gatheredaccount'),
     ]
@@ -12,6 +11,13 @@ class Migration(migrations.Migration):
     operations = [
         migrations.AlterModelOptions(
             name='account',
-            options={'permissions': [('view_accountsecret', 'Can view asset account secret'), ('view_historyaccount', 'Can view asset history account'), ('view_historyaccountsecret', 'Can view asset history account secret'), ('verify_account', 'Can verify account'), ('push_account', 'Can push account')], 'verbose_name': 'Account'},
+            options={'permissions': [
+                ('view_accountsecret', 'Can view asset account secret'),
+                ('view_historyaccount', 'Can view asset history account'),
+                ('view_historyaccountsecret', 'Can view asset history account secret'),
+                ('verify_account', 'Can verify account'),
+                ('push_account', 'Can push account'),
+                ('remove_account', 'Can remove account'),
+            ], 'verbose_name': 'Account'},
         ),
     ]

apps/accounts/migrations/0014_virtualaccount.py

@@ -1,8 +1,9 @@
 # Generated by Django 4.1.10 on 2023-08-01 09:12
 
-from django.db import migrations, models
 import uuid
 
+from django.db import migrations, models
+
 
 class Migration(migrations.Migration):
 
@@ -20,7 +21,7 @@ class Migration(migrations.Migration):
                 ('date_updated', models.DateTimeField(auto_now=True, verbose_name='Date updated')),
                 ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
                 ('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
-                ('alias', models.CharField(choices=[('@INPUT', 'Manual input'), ('@USER', 'Dynamic user'), ('@ANON', 'Anonymous account')], max_length=128, verbose_name='Alias')),
+                ('alias', models.CharField(choices=[('@INPUT', 'Manual input'), ('@USER', 'Dynamic user'), ('@ANON', 'Anonymous account'), ('@SPEC', 'Specified account')], max_length=128, verbose_name='Alias')),
                 ('secret_from_login', models.BooleanField(default=None, null=True, verbose_name='Secret from login')),
             ],
             options={

apps/accounts/models/account.py

@@ -4,6 +4,7 @@ from simple_history.models import HistoricalRecords
 
 from assets.models.base import AbsConnectivity
 from common.utils import lazyproperty
+from labels.mixins import LabeledMixin
 from .base import BaseAccount
 from .mixins import VaultModelMixin
 from ..const import Source
@@ -42,7 +43,7 @@ class AccountHistoricalRecords(HistoricalRecords):
         return super().create_history_model(model, inherited)
 
 
-class Account(AbsConnectivity, BaseAccount):
+class Account(AbsConnectivity, LabeledMixin, BaseAccount):
     asset = models.ForeignKey(
         'assets.Asset', related_name='accounts',
         on_delete=models.CASCADE, verbose_name=_('Asset')
@@ -68,10 +69,15 @@ class Account(AbsConnectivity, BaseAccount):
             ('view_historyaccountsecret', _('Can view asset history account secret')),
             ('verify_account', _('Can verify account')),
             ('push_account', _('Can push account')),
+            ('remove_account', _('Can remove account')),
         ]
 
     def __str__(self):
-        return '{}'.format(self.username)
+        if self.asset_id:
+            host = self.asset.name
+        else:
+            host = 'Dynamic'
+        return '{}({})'.format(self.name, host)
 
     @lazyproperty
     def platform(self):
@@ -95,14 +101,13 @@ class Account(AbsConnectivity, BaseAccount):
         """ 排除自己和以自己为 su-from 的账号 """
         return self.asset.accounts.exclude(id=self.id).exclude(su_from=self)
 
-    @staticmethod
+    def make_account_ansible_vars(self, su_from):
-    def make_account_ansible_vars(su_from):
         var = {
             'ansible_user': su_from.username,
         }
         if not su_from.secret:
             return var
-        var['ansible_password'] = su_from.secret
+        var['ansible_password'] = self.escape_jinja2_syntax(su_from.secret)
         var['ansible_ssh_private_key_file'] = su_from.private_key_path
         return var
 
@@ -119,9 +124,25 @@ class Account(AbsConnectivity, BaseAccount):
         auth['ansible_become'] = True
         auth['ansible_become_method'] = become_method
         auth['ansible_become_user'] = self.username
-        auth['ansible_become_password'] = password
+        auth['ansible_become_password'] = self.escape_jinja2_syntax(password)
         return auth
 
+    @staticmethod
+    def escape_jinja2_syntax(value):
+        if not isinstance(value, str):
+            return value
+
+        def escape(v):
+            v = v.replace('{{', '__TEMP_OPEN_BRACES__') \
+                .replace('}}', '__TEMP_CLOSE_BRACES__')
+
+            v = v.replace('__TEMP_OPEN_BRACES__', '{{ "{{" }}') \
+                .replace('__TEMP_CLOSE_BRACES__', '{{ "}}" }}')
+
+            return v.replace('{%', '{{ "{%" }}').replace('%}', '{{ "%}" }}')
+
+        return escape(value)
+
 
 def replace_history_model_with_mixin():
     """

apps/accounts/models/automations/backup_account.py

@@ -8,7 +8,7 @@ from django.db import models
 from django.db.models import F
 from django.utils.translation import gettext_lazy as _
 
-from accounts.const.automation import AccountBackupType
+from accounts.const import AccountBackupType
 from common.const.choices import Trigger
 from common.db import fields
 from common.db.encoder import ModelJSONFieldEncoder

apps/accounts/models/automations/change_secret.py

@@ -2,7 +2,7 @@ from django.db import models
 from django.utils.translation import gettext_lazy as _
 
 from accounts.const import (
-    AutomationTypes
+    AutomationTypes, ChangeSecretRecordStatusChoice
 )
 from common.db import fields
 from common.db.models import JMSBaseModel
@@ -40,7 +40,10 @@ class ChangeSecretRecord(JMSBaseModel):
     new_secret = fields.EncryptTextField(blank=True, null=True, verbose_name=_('New secret'))
     date_started = models.DateTimeField(blank=True, null=True, verbose_name=_('Date started'))
     date_finished = models.DateTimeField(blank=True, null=True, verbose_name=_('Date finished'))
-    status = models.CharField(max_length=16, default='pending', verbose_name=_('Status'))
+    status = models.CharField(
+        max_length=16, verbose_name=_('Status'),
+        default=ChangeSecretRecordStatusChoice.pending.value
+    )
     error = models.TextField(blank=True, null=True, verbose_name=_('Error'))
 
     class Meta:

apps/accounts/models/base.py

@@ -137,16 +137,13 @@ class BaseAccount(VaultModelMixin, JMSOrgBaseModel):
         else:
             return None
 
-    @property
+    def get_private_key_path(self, path):
-    def private_key_path(self):
         if self.secret_type != SecretType.SSH_KEY \
                 or not self.secret \
                 or not self.private_key:
             return None
-        project_dir = settings.PROJECT_DIR
-        tmp_dir = os.path.join(project_dir, 'tmp')
         key_name = '.' + md5(self.private_key.encode('utf-8')).hexdigest()
-        key_path = os.path.join(tmp_dir, key_name)
+        key_path = os.path.join(path, key_name)
         if not os.path.exists(key_path):
             # https://github.com/ansible/ansible-runner/issues/544
             # ssh requires OpenSSH format keys to have a full ending newline.
@@ -158,6 +155,12 @@ class BaseAccount(VaultModelMixin, JMSOrgBaseModel):
             os.chmod(key_path, 0o400)
         return key_path
 
+    @property
+    def private_key_path(self):
+        project_dir = settings.PROJECT_DIR
+        tmp_dir = os.path.join(project_dir, 'tmp')
+        return self.get_private_key_path(tmp_dir)
+
     def get_private_key(self):
         if not self.private_key:
             return None

apps/accounts/models/template.py

@@ -3,13 +3,14 @@ from django.db.models import Count, Q
 from django.utils import timezone
 from django.utils.translation import gettext_lazy as _
 
+from labels.mixins import LabeledMixin
 from .account import Account
 from .base import BaseAccount, SecretWithRandomMixin
 
 __all__ = ['AccountTemplate', ]
 
 
-class AccountTemplate(BaseAccount, SecretWithRandomMixin):
+class AccountTemplate(LabeledMixin, BaseAccount, SecretWithRandomMixin):
     su_from = models.ForeignKey(
         'self', related_name='su_to', null=True,
         on_delete=models.SET_NULL, verbose_name=_("Su from")

apps/accounts/notifications.py

@@ -1,10 +1,11 @@
 from django.template.loader import render_to_string
 from django.utils.translation import gettext_lazy as _
 
+from accounts.models import ChangeSecretRecord
 from common.tasks import send_mail_attachment_async, upload_backup_to_obj_storage
 from notifications.notifications import UserMessage
-from users.models import User
 from terminal.models.component.storage import ReplayStorage
+from users.models import User
 
 
 class AccountBackupExecutionTaskMsg(object):
@@ -23,8 +24,8 @@ class AccountBackupExecutionTaskMsg(object):
         else:
             return _("{} - The account backup passage task has been completed: "
                      "the encryption password has not been set - "
-                     "please go to personal information -> file encryption password "
+                     "please go to personal information -> Basic file encryption password for preference settings"
-                     "to set the encryption password").format(name)
+                     ).format(name)
 
     def publish(self, attachment_list=None):
         send_mail_attachment_async(
@@ -54,20 +55,23 @@ class AccountBackupByObjStorageExecutionTaskMsg(object):
 class ChangeSecretExecutionTaskMsg(object):
     subject = _('Notification of implementation result of encryption change plan')
 
-    def __init__(self, name: str, user: User):
+    def __init__(self, name: str, user: User, summary):
         self.name = name
         self.user = user
+        self.summary = summary
 
     @property
     def message(self):
         name = self.name
         if self.user.secret_key:
-            return _('{} - The encryption change task has been completed. '
+            default_message = _('{} - The encryption change task has been completed. '
                                 'See the attachment for details').format(name)
+
         else:
-            return _("{} - The encryption change task has been completed: the encryption "
+            default_message = _("{} - The encryption change task has been completed: the encryption "
                                 "password has not been set - please go to personal information -> "
-                     "file encryption password to set the encryption password").format(name)
+                                "set encryption password in preferences").format(name)
+        return self.summary + '\n' + default_message
 
     def publish(self, attachments=None):
         send_mail_attachment_async(
@@ -95,3 +99,35 @@ class GatherAccountChangeMsg(UserMessage):
     def gen_test_msg(cls):
         user = User.objects.first()
         return cls(user, {})
+
+
+class ChangeSecretFailedMsg(UserMessage):
+    subject = _('Change secret or push account failed information')
+
+    def __init__(self, name, execution_id, user, asset_account_errors: list):
+        self.name = name
+        self.execution_id = execution_id
+        self.asset_account_errors = asset_account_errors
+        super().__init__(user)
+
+    def get_html_msg(self) -> dict:
+        context = {
+            'name': self.name,
+            'recipient': self.user,
+            'execution_id': self.execution_id,
+            'asset_account_errors': self.asset_account_errors
+        }
+        message = render_to_string('accounts/change_secret_failed_info.html', context)
+
+        return {
+            'subject': str(self.subject),
+            'message': message
+        }
+
+    @classmethod
+    def gen_test_msg(cls):
+        name = 'test'
+        user = User.objects.first()
+        record = ChangeSecretRecord.objects.first()
+        execution_id = str(record.execution_id)
+        return cls(name, execution_id, user, [])

apps/accounts/permissions.py

@@ -0,0 +1,19 @@
+from rest_framework import permissions
+
+
+def check_permissions(request):
+    act = request.data.get('action')
+    if act == 'push':
+        code = 'accounts.push_account'
+    elif act == 'remove':
+        code = 'accounts.remove_account'
+    else:
+        code = 'accounts.verify_account'
+    return request.user.has_perm(code)
+
+
+class AccountTaskActionPermission(permissions.IsAuthenticated):
+
+    def has_permission(self, request, view):
+        return super().has_permission(request, view) \
+            and check_permissions(request)

apps/accounts/serializers/account/account.py

@@ -10,7 +10,7 @@ from rest_framework.generics import get_object_or_404
 from rest_framework.validators import UniqueTogetherValidator
 
 from accounts.const import SecretType, Source, AccountInvalidPolicy
-from accounts.models import Account, AccountTemplate
+from accounts.models import Account, AccountTemplate, GatheredAccount
 from accounts.tasks import push_accounts_to_assets_task
 from assets.const import Category, AllTypes
 from assets.models import Asset
@@ -58,7 +58,7 @@ class AccountCreateUpdateSerializerMixin(serializers.Serializer):
         for data in initial_data:
             if not data.get('asset') and not self.instance:
                 raise serializers.ValidationError({'asset': UniqueTogetherValidator.missing_message})
-            asset = data.get('asset') or self.instance.asset
+            asset = data.get('asset') or getattr(self.instance, 'asset', None)
             self.from_template_if_need(data)
             self.set_uniq_name_if_need(data, asset)
 
@@ -66,6 +66,9 @@ class AccountCreateUpdateSerializerMixin(serializers.Serializer):
         name = initial_data.get('name')
         if name is not None:
             return
+        request = self.context.get('request')
+        if request and request.method == 'PATCH':
+            return
         if not name:
             name = initial_data.get('username')
         if self.instance and self.instance.name == name:
@@ -238,7 +241,7 @@ class AccountSerializer(AccountCreateUpdateSerializerMixin, BaseAccountSerialize
         queryset = queryset.prefetch_related(
             'asset', 'asset__platform',
             'asset__platform__automation'
-        )
+        ).prefetch_related('labels', 'labels__label')
         return queryset
 
 
@@ -428,8 +431,11 @@ class AssetAccountBulkSerializer(
 
 class AccountSecretSerializer(SecretReadableMixin, AccountSerializer):
     class Meta(AccountSerializer.Meta):
+        fields = AccountSerializer.Meta.fields + ['spec_info']
         extra_kwargs = {
+            **AccountSerializer.Meta.extra_kwargs,
             'secret': {'write_only': False},
+            'spec_info': {'label': _('Spec info')},
         }
 
 
@@ -452,14 +458,20 @@ class AccountHistorySerializer(serializers.ModelSerializer):
 
 class AccountTaskSerializer(serializers.Serializer):
     ACTION_CHOICES = (
-        ('test', 'test'),
         ('verify', 'verify'),
         ('push', 'push'),
+        ('remove', 'remove'),
     )
     action = serializers.ChoiceField(choices=ACTION_CHOICES, write_only=True)
+    assets = serializers.PrimaryKeyRelatedField(
+        queryset=Asset.objects, required=False, allow_empty=True, many=True
+    )
     accounts = serializers.PrimaryKeyRelatedField(
         queryset=Account.objects, required=False, allow_empty=True, many=True
     )
+    gather_accounts = serializers.PrimaryKeyRelatedField(
+        queryset=GatheredAccount.objects, required=False, allow_empty=True, many=True
+    )
     task = serializers.CharField(read_only=True)
     params = serializers.JSONField(
         decoder=None, encoder=None, required=False,

apps/accounts/serializers/account/base.py

@@ -5,6 +5,7 @@ from rest_framework import serializers
 from accounts.const import SecretType
 from accounts.models import BaseAccount
 from accounts.utils import validate_password_for_ansible, validate_ssh_key
+from common.serializers import ResourceLabelsMixin
 from common.serializers.fields import EncryptedField, LabeledChoiceField
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 
@@ -60,22 +61,20 @@ class AuthValidateMixin(serializers.Serializer):
         return super().update(instance, validated_data)
 
 
-class BaseAccountSerializer(AuthValidateMixin, BulkOrgResourceModelSerializer):
+class BaseAccountSerializer(AuthValidateMixin, ResourceLabelsMixin, BulkOrgResourceModelSerializer):
-
     class Meta:
         model = BaseAccount
         fields_mini = ['id', 'name', 'username']
         fields_small = fields_mini + [
             'secret_type', 'secret', 'passphrase',
-            'privileged', 'is_active', 'spec_info',
+            'privileged', 'is_active',
         ]
         fields_other = ['created_by', 'date_created', 'date_updated', 'comment']
-        fields = fields_small + fields_other
+        fields = fields_small + fields_other + ['labels']
         read_only_fields = [
-            'spec_info', 'date_verified', 'created_by', 'date_created',
+            'date_verified', 'created_by', 'date_created',
         ]
         extra_kwargs = {
-            'spec_info': {'label': _('Spec info')},
             'username': {'help_text': _(
                 "Tip: If no username is required for authentication, fill in `null`, "
                 "If AD account, like `username@domain`"

apps/accounts/serializers/account/template.py

@@ -15,6 +15,9 @@ class PasswordRulesSerializer(serializers.Serializer):
     uppercase = serializers.BooleanField(default=True, label=_('Uppercase'))
     digit = serializers.BooleanField(default=True, label=_('Digit'))
     symbol = serializers.BooleanField(default=True, label=_('Special symbol'))
+    exclude_symbols = serializers.CharField(
+        default='', allow_blank=True, max_length=16, label=_('Exclude symbol')
+    )
 
 
 class AccountTemplateSerializer(BaseAccountSerializer):
@@ -32,6 +35,7 @@ class AccountTemplateSerializer(BaseAccountSerializer):
             'su_from'
         ]
         extra_kwargs = {
+            **BaseAccountSerializer.Meta.extra_kwargs,
             'secret_strategy': {'help_text': _('Secret generation strategy for account creation')},
             'auto_push': {'help_text': _('Whether to automatically push the account to the asset')},
             'platforms': {
@@ -61,6 +65,9 @@ class AccountTemplateSerializer(BaseAccountSerializer):
 
 class AccountTemplateSecretSerializer(SecretReadableMixin, AccountTemplateSerializer):
     class Meta(AccountTemplateSerializer.Meta):
+        fields = AccountTemplateSerializer.Meta.fields + ['spec_info']
         extra_kwargs = {
+            **AccountTemplateSerializer.Meta.extra_kwargs,
             'secret': {'write_only': False},
+            'spec_info': {'label': _('Spec info')},
         }

apps/accounts/serializers/automations/base.py

@@ -21,6 +21,7 @@ __all__ = [
 class BaseAutomationSerializer(PeriodTaskSerializerMixin, BulkOrgResourceModelSerializer):
     assets = ObjectRelatedField(many=True, required=False, queryset=Asset.objects, label=_('Assets'))
     nodes = ObjectRelatedField(many=True, required=False, queryset=Node.objects, label=_('Nodes'))
+    is_periodic = serializers.BooleanField(default=False, required=False, label=_("Periodic perform"))
 
     class Meta:
         read_only_fields = [

apps/accounts/serializers/automations/change_secret.py

@@ -4,7 +4,8 @@ from django.utils.translation import gettext_lazy as _
 from rest_framework import serializers
 
 from accounts.const import (
-    AutomationTypes, SecretType, SecretStrategy, SSHKeyStrategy
+    AutomationTypes, SecretType, SecretStrategy,
+    SSHKeyStrategy, ChangeSecretRecordStatusChoice
 )
 from accounts.models import (
     Account, ChangeSecretAutomation,
@@ -21,6 +22,7 @@ logger = get_logger(__file__)
 __all__ = [
     'ChangeSecretAutomationSerializer',
     'ChangeSecretRecordSerializer',
+    'ChangeSecretRecordViewSecretSerializer',
     'ChangeSecretRecordBackUpSerializer',
     'ChangeSecretUpdateAssetSerializer',
     'ChangeSecretUpdateNodeSerializer',
@@ -104,7 +106,10 @@ class ChangeSecretAutomationSerializer(AuthValidateMixin, BaseAutomationSerializ
 class ChangeSecretRecordSerializer(serializers.ModelSerializer):
     is_success = serializers.SerializerMethodField(label=_('Is success'))
     asset = ObjectRelatedField(queryset=Asset.objects, label=_('Asset'))
-    account = ObjectRelatedField(queryset=Account.objects, label=_('Account'))
+    account = ObjectRelatedField(
+        queryset=Account.objects, label=_('Account'),
+        attrs=("id", "name", "username")
+    )
     execution = ObjectRelatedField(
         queryset=AutomationExecution.objects, label=_('Automation task execution')
     )
@@ -119,7 +124,16 @@ class ChangeSecretRecordSerializer(serializers.ModelSerializer):
 
     @staticmethod
     def get_is_success(obj):
-        return obj.status == 'success'
+        return obj.status == ChangeSecretRecordStatusChoice.success.value
+
+
+class ChangeSecretRecordViewSecretSerializer(serializers.ModelSerializer):
+    class Meta:
+        model = ChangeSecretRecord
+        fields = [
+            'id', 'old_secret', 'new_secret',
+        ]
+        read_only_fields = fields
 
 
 class ChangeSecretRecordBackUpSerializer(serializers.ModelSerializer):
@@ -145,7 +159,7 @@ class ChangeSecretRecordBackUpSerializer(serializers.ModelSerializer):
 
     @staticmethod
     def get_is_success(obj):
-        if obj.status == 'success':
+        if obj.status == ChangeSecretRecordStatusChoice.success.value:
             return _("Success")
         return _("Failed")
 

apps/accounts/signal_handlers.py

@@ -21,7 +21,8 @@ def on_account_pre_save(sender, instance, **kwargs):
     if instance.version == 0:
         instance.version = 1
     else:
-        instance.version = instance.history.count()
+        history_account = instance.history.first()
+        instance.version = history_account.version + 1 if history_account else 0
 
 
 @merge_delay_run(ttl=5)
@@ -62,7 +63,7 @@ def create_accounts_activities(account, action='create'):
 def on_account_create_by_template(sender, instance, created=False, **kwargs):
     if not created or instance.source != 'template':
         return
-    push_accounts_if_need(accounts=(instance,))
+    push_accounts_if_need.delay(accounts=(instance,))
     create_accounts_activities(instance, action='create')
 
 

apps/accounts/tasks/__init__.py

@@ -2,5 +2,6 @@ from .automation import *
 from .backup_account import *
 from .gather_accounts import *
 from .push_account import *
+from .remove_account import *
 from .template import *
 from .verify_account import *

apps/accounts/tasks/automation.py

@@ -36,14 +36,14 @@ def execute_account_automation_task(pid, trigger, tp):
         instance.execute(trigger)
 
 
-def record_task_activity_callback(self, record_id, *args, **kwargs):
+def record_task_activity_callback(self, record_ids, *args, **kwargs):
     from accounts.models import ChangeSecretRecord
     with tmp_to_root_org():
-        record = get_object_or_none(ChangeSecretRecord, id=record_id)
+        records = ChangeSecretRecord.objects.filter(id__in=record_ids)
-    if not record:
+    if not records:
         return
-    resource_ids = [record.id]
+    resource_ids = [str(i.id) for i in records]
-    org_id = record.execution.org_id
+    org_id = records[0].execution.org_id
     return resource_ids, org_id
 
 
@@ -51,22 +51,26 @@ def record_task_activity_callback(self, record_id, *args, **kwargs):
     queue='ansible', verbose_name=_('Execute automation record'),
     activity_callback=record_task_activity_callback
 )
-def execute_automation_record_task(record_id, tp):
+def execute_automation_record_task(record_ids, tp):
     from accounts.models import ChangeSecretRecord
+    task_name = gettext_noop('Execute automation record')
+
     with tmp_to_root_org():
-        instance = get_object_or_none(ChangeSecretRecord, pk=record_id)
+        records = ChangeSecretRecord.objects.filter(id__in=record_ids)
-    if not instance:
+
-        logger.error("No automation record found: {}".format(record_id))
+    if not records:
+        logger.error('No automation record found: {}'.format(record_ids))
         return
 
-    task_name = gettext_noop('Execute automation record')
+    record = records[0]
+    record_map = {f'{record.asset_id}-{record.account_id}': str(record.id) for record in records}
     task_snapshot = {
-        'secret': instance.new_secret,
-        'secret_type': instance.execution.snapshot.get('secret_type'),
-        'accounts': [str(instance.account_id)],
-        'assets': [str(instance.asset_id)],
         'params': {},
-        'record_id': record_id,
+        'record_map': record_map,
+        'secret': record.new_secret,
+        'secret_type': record.execution.snapshot.get('secret_type'),
+        'assets': [str(instance.asset_id) for instance in records],
+        'accounts': [str(instance.account_id) for instance in records],
     }
-    with tmp_to_org(instance.execution.org_id):
+    with tmp_to_org(record.execution.org_id):
         quickstart_automation_by_snapshot(task_name, tp, task_snapshot)

apps/accounts/tasks/remove_account.py

@@ -0,0 +1,77 @@
+import uuid
+from collections import defaultdict
+
+from celery import shared_task, current_task
+from django.conf import settings
+from django.db.models import Count
+from django.utils.translation import gettext_noop, gettext_lazy as _
+
+from accounts.const import AutomationTypes
+from accounts.models import Account
+from accounts.tasks.common import quickstart_automation_by_snapshot
+from audits.const import ActivityChoices
+from common.const.crontab import CRONTAB_AT_AM_TWO
+from common.utils import get_logger
+from ops.celery.decorator import register_as_period_task
+from orgs.utils import tmp_to_root_org
+
+logger = get_logger(__file__)
+
+__all__ = ['remove_accounts_task']
+
+
+@shared_task(
+    queue="ansible", verbose_name=_('Remove account'),
+    activity_callback=lambda self, gather_account_ids, *args, **kwargs: (gather_account_ids, None)
+)
+def remove_accounts_task(gather_account_ids):
+    from accounts.models import GatheredAccount
+
+    gather_accounts = GatheredAccount.objects.filter(
+        id__in=gather_account_ids
+    )
+    task_name = gettext_noop("Remove account")
+
+    task_snapshot = {
+        'assets': [str(i.asset_id) for i in gather_accounts],
+        'gather_accounts': [str(i.id) for i in gather_accounts],
+    }
+
+    tp = AutomationTypes.remove_account
+    quickstart_automation_by_snapshot(task_name, tp, task_snapshot)
+
+
+@shared_task(verbose_name=_('Clean historical accounts'))
+@register_as_period_task(crontab=CRONTAB_AT_AM_TWO)
+@tmp_to_root_org()
+def clean_historical_accounts():
+    from audits.signal_handlers import create_activities
+    print("Clean historical accounts start.")
+    if settings.HISTORY_ACCOUNT_CLEAN_LIMIT >= 999:
+        return
+    limit = settings.HISTORY_ACCOUNT_CLEAN_LIMIT
+
+    history_ids_to_be_deleted = []
+    history_model = Account.history.model
+    history_id_mapper = defaultdict(list)
+
+    ids = history_model.objects.values('id').annotate(count=Count('id')) \
+        .filter(count__gte=limit).values_list('id', flat=True)
+
+    if not ids:
+        return
+
+    for i in history_model.objects.filter(id__in=ids):
+        _id = str(i.id)
+        history_id_mapper[_id].append(i.history_id)
+
+    for history_ids in history_id_mapper.values():
+        history_ids_to_be_deleted.extend(history_ids[limit:])
+    history_qs = history_model.objects.filter(history_id__in=history_ids_to_be_deleted)
+
+    resource_ids = list(history_qs.values_list('history_id', flat=True))
+    history_qs.delete()
+
+    task_id = current_task.request.id if current_task else str(uuid.uuid4())
+    detail = gettext_noop('Remove historical accounts that are out of range.')
+    create_activities(resource_ids, detail, task_id, action=ActivityChoices.task, org_id='')

apps/accounts/tasks/template.py

@@ -29,7 +29,8 @@ def template_sync_related_accounts(template_id, user_id=None):
     name = template.name
     username = template.username
     secret_type = template.secret_type
-    print(f'\033[32m>>> 开始同步模版名称、用户名、密钥类型到相关联的账号 ({datetime.now().strftime("%Y-%m-%d %H:%M:%S")})')
+    print(
+        f'\033[32m>>> 开始同步模板名称、用户名、密钥类型到相关联的账号 ({datetime.now().strftime("%Y-%m-%d %H:%M:%S")})')
     with tmp_to_org(org_id):
         for account in accounts:
             account.name = name

apps/accounts/templates/accounts/asset_account_change_info.html

@@ -1,10 +1,10 @@
 {% load i18n %}
 
-
+<h3>{% trans 'Gather account change information' %}</h3>
 <table style="width: 100%; border-collapse: collapse; max-width: 100%; text-align: left; margin-top: 20px;">
     <caption></caption>
     <tr style="background-color: #f2f2f2;">
-        <th style="border: 1px solid #ddd; padding: 10px; font-weight: bold;">{% trans 'Asset' %}</th>
+        <th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Asset' %}</th>
         <th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Added account' %}</th>
         <th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Deleted account' %}</th>
     </tr>

apps/accounts/templates/accounts/change_secret_failed_info.html

@@ -0,0 +1,36 @@
+{% load i18n %}
+
+<h3>{% trans 'Task name' %}: {{ name }}</h3>
+<h3>{% trans 'Task execution id' %}: {{ execution_id }}</h3>
+<p>{% trans 'Respectful' %} {{ recipient }}</p>
+<p>{% trans 'Hello! The following is the failure of changing the password of your assets or pushing the account. Please check and handle it in time.' %}</p>
+<table style="width: 100%; border-collapse: collapse; max-width: 100%; text-align: left; margin-top: 20px;">
+    <caption></caption>
+    <thead>
+    <tr style="background-color: #f2f2f2;">
+        <th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Asset' %}</th>
+        <th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Account' %}</th>
+        <th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Error' %}</th>
+    </tr>
+    </thead>
+    <tbody>
+    {% for asset_name, account_username, error in asset_account_errors %}
+        <tr>
+            <td style="border: 1px solid #ddd; padding: 10px;">{{ asset_name }}</td>
+            <td style="border: 1px solid #ddd; padding: 10px;">{{ account_username }}</td>
+            <td style="border: 1px solid #ddd; padding: 10px;">
+                <div style="
+                max-width: 90%;
+                white-space: nowrap;
+                overflow: hidden;
+                text-overflow: ellipsis;
+                display: block;"
+                     title="{{ error }}"
+                >
+                    {{ error }}
+                </div>
+            </td>
+        </tr>
+    {% endfor %}
+    </tbody>
+</table>

apps/accounts/utils.py

@@ -30,7 +30,8 @@ class SecretGenerator:
             'lower': rules['lowercase'],
             'upper': rules['uppercase'],
             'digit': rules['digit'],
-            'special_char': rules['symbol']
+            'special_char': rules['symbol'],
+            'exclude_chars': rules.get('exclude_symbols', ''),
         }
         return random_string(**rules)
 
@@ -46,18 +47,10 @@ class SecretGenerator:
 
 def validate_password_for_ansible(password):
     """ 校验 Ansible 不支持的特殊字符 """
-    # validate password contains left double curly bracket
+    if password.startswith('{{') and password.endswith('}}'):
-    # check password not contains `{{`
+        raise serializers.ValidationError(
-    # Ansible 推送的时候不支持
+            _('If the password starts with {{` and ends with }} `, then the password is not allowed.')
-    if '{{' in password or '}}' in password:
+        )
-        raise serializers.ValidationError(_('Password can not contains `{{` or `}}`'))
-    if '{%' in password or '%}' in password:
-        raise serializers.ValidationError(_('Password can not contains `{%` or `%}`'))
-    # Ansible Windows 推送的时候不支持
-    # if "'" in password:
-    #     raise serializers.ValidationError(_("Password can not contains `'` "))
-    # if '"' in password:
-    #     raise serializers.ValidationError(_('Password can not contains `"` '))
 
 
 def validate_ssh_key(ssh_key, passphrase=None):

apps/acls/api/command_acl.py

@@ -11,7 +11,7 @@ __all__ = ['CommandFilterACLViewSet', 'CommandGroupViewSet']
 class CommandGroupViewSet(OrgBulkModelViewSet):
     model = models.CommandGroup
     filterset_fields = ('name', 'command_filters')
-    search_fields = filterset_fields
+    search_fields = ('name',)
     serializer_class = serializers.CommandGroupSerializer
 
 

apps/acls/notifications.py

@@ -1,6 +1,7 @@
 from django.template.loader import render_to_string
 from django.utils.translation import gettext_lazy as _
 
+from accounts.models import Account
 from assets.models import Asset
 from audits.models import UserLoginLog
 from notifications.notifications import UserMessage
@@ -16,12 +17,11 @@ class UserLoginReminderMsg(UserMessage):
 
     def get_html_msg(self) -> dict:
         user_log = self.user_log
-
         context = {
             'ip': user_log.ip,
             'city': user_log.city,
             'username': user_log.username,
-            'recipient': self.user.username,
+            'recipient': self.user,
             'user_agent': user_log.user_agent,
         }
         message = render_to_string('acls/user_login_reminder.html', context)
@@ -41,18 +41,21 @@ class UserLoginReminderMsg(UserMessage):
 class AssetLoginReminderMsg(UserMessage):
     subject = _('Asset login reminder')
 
-    def __init__(self, user, asset: Asset, login_user: User, account_username):
+    def __init__(self, user, asset: Asset, login_user: User, account: Account, input_username):
         self.asset = asset
         self.login_user = login_user
-        self.account_username = account_username
+        self.account = account
+        self.input_username = input_username
         super().__init__(user)
 
     def get_html_msg(self) -> dict:
         context = {
-            'recipient': self.user.username,
+            'recipient': self.user,
             'username': self.login_user.username,
+            'name': self.login_user.name,
             'asset': str(self.asset),
-            'account': self.account_username,
+            'account': self.input_username,
+            'account_name': self.account.name,
         }
         message = render_to_string('acls/asset_login_reminder.html', context)
 

apps/acls/templates/acls/asset_login_reminder.html

@@ -1,10 +1,10 @@
 {% load i18n %}
 
-<h3>{% trans 'Respectful' %}{{ recipient }}，</h3>
+<h3>{% trans 'Respectful' %}: {{ recipient.name }}[{{ recipient.username }}]</h3>
 <hr>
-<p><strong>{% trans 'Username' %}:</strong> [{{ username }}]</p>
+<p><strong>{% trans 'User' %}:</strong> [{{ name }}({{ username }})]</p>
 <p><strong>{% trans 'Assets' %}:</strong> [{{ asset }}]</p>
-<p><strong>{% trans 'Account' %}:</strong> [{{ account }}]</p>
+<p><strong>{% trans 'Account' %}:</strong> [{{ account_name }}({{ account }})]</p>
 <hr>
 
 <p>{% trans 'The user has just logged in to the asset. Please ensure that this is an authorized operation. If you suspect that this is an unauthorized access, please take appropriate measures immediately.' %}</p>

apps/acls/templates/acls/user_login_reminder.html

@@ -1,8 +1,8 @@
 {% load i18n %}
 
-<h3>{% trans 'Respectful' %}{{ recipient }}，</h3>
+<h3>{% trans 'Respectful' %}: {{ recipient.name }}[{{ recipient.username }}]</h3>
 <hr>
-<p><strong>{% trans 'Username' %}:</strong> [{{ username }}]</p>
+<p><strong>{% trans 'User' %}:</strong> [{{ username }}]</p>
 <p><strong>IP:</strong> [{{ ip }}]</p>
 <p><strong>{% trans 'Login city' %}:</strong> [{{ city }}]</p>
 <p><strong>{% trans 'User agent' %}:</strong> [{{ user_agent }}]</p>

apps/assets/api/__init__.py

@@ -2,7 +2,6 @@ from .asset import *
 from .category import *
 from .domain import *
 from .favorite_asset import *
-from .label import *
 from .mixin import *
 from .node import *
 from .platform import *

apps/assets/api/asset/asset.py

@@ -3,7 +3,6 @@
 from collections import defaultdict
 
 import django_filters
-from django.db.models import Q
 from django.shortcuts import get_object_or_404
 from django.utils.translation import gettext as _
 from rest_framework import status
@@ -14,7 +13,7 @@ from rest_framework.status import HTTP_200_OK
 from accounts.tasks import push_accounts_to_assets_task, verify_accounts_connectivity_task
 from assets import serializers
 from assets.exceptions import NotSupportedTemporarilyError
-from assets.filters import IpInFilterBackend, LabelFilterBackend, NodeFilterBackend
+from assets.filters import IpInFilterBackend, NodeFilterBackend
 from assets.models import Asset, Gateway, Platform, Protocol
 from assets.tasks import test_assets_connectivity_manual, update_assets_hardware_info_manual
 from common.api import SuggestionMixin
@@ -22,7 +21,6 @@ from common.drf.filters import BaseFilterSet, AttrRulesFilterBackend
 from common.utils import get_logger, is_uuid
 from orgs.mixins import generics
 from orgs.mixins.api import OrgBulkModelViewSet
-from ..mixin import NodeFilterMixin
 from ...notifications import BulkUpdatePlatformSkipAssetUserMsg
 
 logger = get_logger(__file__)
@@ -33,8 +31,8 @@ __all__ = [
 
 
 class AssetFilterSet(BaseFilterSet):
-    labels = django_filters.CharFilter(method='filter_labels')
     platform = django_filters.CharFilter(method='filter_platform')
+    exclude_platform = django_filters.CharFilter(field_name="platform__name", lookup_expr='exact', exclude=True)
     domain = django_filters.CharFilter(method='filter_domain')
     type = django_filters.CharFilter(field_name="platform__type", lookup_expr="exact")
     category = django_filters.CharFilter(field_name="platform__category", lookup_expr="exact")
@@ -64,7 +62,7 @@ class AssetFilterSet(BaseFilterSet):
     class Meta:
         model = Asset
         fields = [
-            "id", "name", "address", "is_active", "labels",
+            "id", "name", "address", "is_active",
             "type", "category", "platform",
         ]
 
@@ -87,25 +85,15 @@ class AssetFilterSet(BaseFilterSet):
         value = value.split(',')
         return queryset.filter(protocols__name__in=value).distinct()
 
-    @staticmethod
-    def filter_labels(queryset, name, value):
-        if ':' in value:
-            n, v = value.split(':', 1)
-            queryset = queryset.filter(labels__name=n, labels__value=v)
-        else:
-            q = Q(labels__name__contains=value) | Q(labels__value__contains=value)
-            queryset = queryset.filter(q).distinct()
-        return queryset
 
-
+class AssetViewSet(SuggestionMixin, OrgBulkModelViewSet):
-class AssetViewSet(SuggestionMixin, NodeFilterMixin, OrgBulkModelViewSet):
     """
     API endpoint that allows Asset to be viewed or edited.
     """
     model = Asset
     filterset_class = AssetFilterSet
     search_fields = ("name", "address", "comment")
-    ordering_fields = ('name', 'connectivity', 'platform', 'date_updated')
+    ordering_fields = ('name', 'address', 'connectivity', 'platform', 'date_updated', 'date_created')
     serializer_classes = (
         ("default", serializers.AssetSerializer),
         ("platform", serializers.PlatformSerializer),
@@ -121,14 +109,12 @@ class AssetViewSet(SuggestionMixin, NodeFilterMixin, OrgBulkModelViewSet):
         ("sync_platform_protocols", "assets.change_asset"),
     )
     extra_filter_backends = [
-        LabelFilterBackend, IpInFilterBackend,
+        IpInFilterBackend,
         NodeFilterBackend, AttrRulesFilterBackend
     ]
 
     def get_queryset(self):
-        queryset = super().get_queryset() \
+        queryset = super().get_queryset()
-            .prefetch_related('nodes', 'protocols') \
-            .select_related('platform', 'domain')
         if queryset.model is not Asset:
             queryset = queryset.select_related('asset_ptr')
         return queryset

apps/assets/api/asset/permission.py

@@ -48,7 +48,7 @@ class AssetPermUserListApi(BaseAssetPermUserOrUserGroupListApi):
 
     def get_queryset(self):
         perms = self.get_asset_related_perms()
-        users = User.objects.filter(
+        users = User.get_queryset().filter(
             Q(assetpermissions__in=perms) | Q(groups__assetpermissions__in=perms)
         ).distinct()
         return users

apps/assets/api/domain.py

@@ -19,15 +19,19 @@ class DomainViewSet(OrgBulkModelViewSet):
     model = Domain
     filterset_fields = ("name",)
     search_fields = filterset_fields
-    ordering = ('name',)
+    serializer_classes = {
+        'default': serializers.DomainSerializer,
+        'list': serializers.DomainListSerializer,
+    }
 
     def get_serializer_class(self):
         if self.request.query_params.get('gateway'):
             return serializers.DomainWithGatewaySerializer
-        return serializers.DomainSerializer
+        return super().get_serializer_class()
 
-    def get_queryset(self):
+    def partial_update(self, request, *args, **kwargs):
-        return super().get_queryset().prefetch_related('assets')
+        kwargs['partial'] = True
+        return self.update(request, *args, **kwargs)
 
 
 class GatewayViewSet(HostViewSet):

apps/assets/api/label.py

@@ -1,43 +0,0 @@
-# ~*~ coding: utf-8 ~*~
-# Copyright (C) 2014-2018 Beijing DuiZhan Technology Co.,Ltd. All Rights Reserved.
-#
-# Licensed under the GNU General Public License v2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#    http://www.gnu.org/licenses/gpl-2.0.html
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from django.db.models import Count
-
-from common.utils import get_logger
-from orgs.mixins.api import OrgBulkModelViewSet
-from ..models import Label
-from .. import serializers
-
-
-logger = get_logger(__file__)
-__all__ = ['LabelViewSet']
-
-
-class LabelViewSet(OrgBulkModelViewSet):
-    model = Label
-    filterset_fields = ("name", "value")
-    search_fields = filterset_fields
-    serializer_class = serializers.LabelSerializer
-
-    def list(self, request, *args, **kwargs):
-        if request.query_params.get("distinct"):
-            self.serializer_class = serializers.LabelDistinctSerializer
-            self.queryset = self.queryset.values("name").distinct()
-        return super().list(request, *args, **kwargs)
-
-    def get_queryset(self):
-        self.queryset = Label.objects.prefetch_related(
-            'assets').annotate(asset_count=Count("assets"))
-        return self.queryset

apps/assets/api/mixin.py

@@ -2,7 +2,7 @@ from typing import List
 
 from rest_framework.request import Request
 
-from assets.models import Node, Protocol
+from assets.models import Node, Platform, Protocol
 from assets.utils import get_node_from_request, is_query_node_all_assets
 from common.utils import lazyproperty, timeit
 
@@ -71,37 +71,49 @@ class SerializeToTreeNodeMixin:
             return 'file'
 
     @timeit
-    def serialize_assets(self, assets, node_key=None, pid=None):
+    def serialize_assets(self, assets, node_key=None, get_pid=None):
-        if node_key is None:
+        if not get_pid and not node_key:
-            get_pid = lambda asset: getattr(asset, 'parent_key', '')
+            get_pid = lambda asset, platform: getattr(asset, 'parent_key', '')
-        else:
+
-            get_pid = lambda asset: node_key
         sftp_asset_ids = Protocol.objects.filter(name='sftp') \
             .values_list('asset_id', flat=True)
-        sftp_asset_ids = list(sftp_asset_ids)
+        sftp_asset_ids = set(sftp_asset_ids)
-        data = [
+        platform_map = {p.id: p for p in Platform.objects.all()}
-            {
+
+        data = []
+        root_assets_count = 0
+        for asset in assets:
+            platform = platform_map.get(asset.platform_id)
+            if not platform:
+                continue
+            pid = node_key or get_pid(asset, platform)
+            if not pid:
+                continue
+            # 根节点最多显示 1000 个资产
+            if pid.isdigit():
+                if root_assets_count > 1000:
+                    continue
+                root_assets_count += 1
+            data.append({
                 'id': str(asset.id),
                 'name': asset.name,
-                'title': f'{asset.address}\n{asset.comment}',
+                'title': f'{asset.address}\n{asset.comment}'.strip(),
-                'pId': pid or get_pid(asset),
+                'pId': pid,
                 'isParent': False,
                 'open': False,
-                'iconSkin': self.get_icon(asset),
+                'iconSkin': self.get_icon(platform),
                 'chkDisabled': not asset.is_active,
                 'meta': {
                     'type': 'asset',
                     'data': {
-                        'platform_type': asset.platform.type,
+                        'platform_type': platform.type,
                         'org_name': asset.org_name,
                         'sftp': asset.id in sftp_asset_ids,
                         'name': asset.name,
                         'address': asset.address
                     },
                 }
-            }
+            })
-            for asset in assets
-        ]
         return data
 
 

apps/assets/api/node.py

@@ -22,6 +22,7 @@ from orgs.utils import current_org
 from rbac.permissions import RBACPermission
 from .. import serializers
 from ..models import Node
+from ..signal_handlers import update_nodes_assets_amount
 from ..tasks import (
     update_node_assets_hardware_info_manual,
     test_node_assets_connectivity_manual,
@@ -94,6 +95,7 @@ class NodeAddChildrenApi(generics.UpdateAPIView):
         children = Node.objects.filter(id__in=node_ids)
         for node in children:
             node.parent = instance
+        update_nodes_assets_amount.delay(ttl=5, node_ids=(instance.id,))
         return Response("OK")
 
 

apps/assets/api/platform.py

@@ -21,6 +21,7 @@ class AssetPlatformViewSet(JMSModelViewSet):
     }
     filterset_fields = ['name', 'category', 'type']
     search_fields = ['name']
+    ordering = ['-internal', 'name']
     rbac_perms = {
         'categories': 'assets.view_platform',
         'type_constraints': 'assets.view_platform',
@@ -29,7 +30,10 @@ class AssetPlatformViewSet(JMSModelViewSet):
     }
 
     def get_queryset(self):
-        queryset = super().get_queryset()
+        # 因为没有走分页逻辑，所以需要这里 prefetch
+        queryset = super().get_queryset().prefetch_related(
+            'protocols', 'automation', 'labels', 'labels__label',
+        )
         queryset = queryset.filter(type__in=AllTypes.get_types_values())
         return queryset
 

apps/assets/api/tree.py

@@ -126,6 +126,8 @@ class NodeChildrenAsTreeApi(SerializeToTreeNodeMixin, NodeChildrenApi):
         include_assets = self.request.query_params.get('assets', '0') == '1'
         if not self.instance or not include_assets:
             return Asset.objects.none()
+        if self.instance.is_org_root():
+            return Asset.objects.none()
         if query_all:
             assets = self.instance.get_all_assets()
         else:

apps/assets/automations/__init__.py

@@ -1,2 +1,2 @@
 from .endpoint import ExecutionManager
-from .methods import platform_automation_methods, filter_platform_methods
+from .methods import platform_automation_methods, filter_platform_methods, sorted_methods

apps/assets/automations/base/manager.py

@@ -12,11 +12,70 @@ from sshtunnel import SSHTunnelForwarder
 
 from assets.automations.methods import platform_automation_methods
 from common.utils import get_logger, lazyproperty, is_openssh_format_key, ssh_pubkey_gen
-from ops.ansible import JMSInventory, PlaybookRunner, DefaultCallback
+from ops.ansible import JMSInventory, DefaultCallback, SuperPlaybookRunner
+from ops.ansible.interface import interface
 
 logger = get_logger(__name__)
 
 
+class SSHTunnelManager:
+    def __init__(self, *args, **kwargs):
+        self.gateway_servers = dict()
+
+    @staticmethod
+    def file_to_json(path):
+        with open(path, 'r') as f:
+            d = json.load(f)
+        return d
+
+    @staticmethod
+    def json_to_file(path, data):
+        with open(path, 'w') as f:
+            json.dump(data, f, indent=4, sort_keys=True)
+
+    def local_gateway_prepare(self, runner):
+        info = self.file_to_json(runner.inventory)
+        servers, not_valid = [], []
+        for k, host in info['all']['hosts'].items():
+            jms_asset, jms_gateway = host.get('jms_asset'), host.get('gateway')
+            if not jms_gateway:
+                continue
+            try:
+                server = SSHTunnelForwarder(
+                    (jms_gateway['address'], jms_gateway['port']),
+                    ssh_username=jms_gateway['username'],
+                    ssh_password=jms_gateway['secret'],
+                    ssh_pkey=jms_gateway['private_key_path'],
+                    remote_bind_address=(jms_asset['address'], jms_asset['port'])
+                )
+                server.start()
+            except Exception as e:
+                err_msg = 'Gateway is not active: %s' % jms_asset.get('name', '')
+                print(f'\033[31m {err_msg} 原因: {e} \033[0m\n')
+                not_valid.append(k)
+            else:
+                local_bind_port = server.local_bind_port
+
+                host['ansible_host'] = jms_asset['address'] = host[
+                    'login_host'] = interface.get_gateway_proxy_host()
+                host['ansible_port'] = jms_asset['port'] = host['login_port'] = local_bind_port
+                servers.append(server)
+
+        # 网域不可连接的，就不继续执行此资源的后续任务了
+        for a in set(not_valid):
+            info['all']['hosts'].pop(a)
+        self.json_to_file(runner.inventory, info)
+        self.gateway_servers[runner.id] = servers
+
+    def local_gateway_clean(self, runner):
+        servers = self.gateway_servers.get(runner.id, [])
+        for s in servers:
+            try:
+                s.stop()
+            except Exception:
+                pass
+
+
 class PlaybookCallback(DefaultCallback):
     def playbook_on_stats(self, event_data, **kwargs):
         super().playbook_on_stats(event_data, **kwargs)
@@ -37,7 +96,6 @@ class BasePlaybookManager:
         # 根据执行方式就行分组, 不同资产的改密、推送等操作可能会使用不同的执行方式
         # 然后根据执行方式分组, 再根据 bulk_size 分组, 生成不同的 playbook
         self.playbooks = []
-        self.gateway_servers = dict()
         params = self.execution.snapshot.get('params')
         self.params = params or {}
 
@@ -157,22 +215,19 @@ class BasePlaybookManager:
             os.chmod(key_path, 0o400)
         return key_path
 
-    def generate_inventory(self, platformed_assets, inventory_path):
+    def generate_inventory(self, platformed_assets, inventory_path, protocol):
         inventory = JMSInventory(
             assets=platformed_assets,
             account_prefer=self.ansible_account_prefer,
             account_policy=self.ansible_account_policy,
             host_callback=self.host_callback,
             task_type=self.__class__.method_type(),
+            protocol=protocol,
         )
         inventory.write_to_file(inventory_path)
 
-    def generate_playbook(self, platformed_assets, platform, sub_playbook_dir):
+    @staticmethod
-        method_id = getattr(platform.automation, '{}_method'.format(self.__class__.method_type()))
+    def generate_playbook(method, sub_playbook_dir):
-        method = self.method_id_meta_mapper.get(method_id)
-        if not method:
-            logger.error("Method not found: {}".format(method_id))
-            return
         method_playbook_dir_path = method['dir']
         sub_playbook_path = os.path.join(sub_playbook_dir, 'project', 'main.yml')
         shutil.copytree(method_playbook_dir_path, os.path.dirname(sub_playbook_path))
@@ -204,12 +259,20 @@ class BasePlaybookManager:
                 sub_dir = '{}_{}'.format(platform.name, i)
                 playbook_dir = os.path.join(self.runtime_dir, sub_dir)
                 inventory_path = os.path.join(self.runtime_dir, sub_dir, 'hosts.json')
-                self.generate_inventory(_assets, inventory_path)
+
-                playbook_path = self.generate_playbook(_assets, platform, playbook_dir)
+                method_id = getattr(platform.automation, '{}_method'.format(self.__class__.method_type()))
+                method = self.method_id_meta_mapper.get(method_id)
+
+                if not method:
+                    logger.error("Method not found: {}".format(method_id))
+                    continue
+                protocol = method.get('protocol')
+                self.generate_inventory(_assets, inventory_path, protocol)
+                playbook_path = self.generate_playbook(method, playbook_dir)
                 if not playbook_path:
                     continue
 
-                runer = PlaybookRunner(
+                runer = SuperPlaybookRunner(
                     inventory_path,
                     playbook_path,
                     self.runtime_dir,
@@ -237,80 +300,28 @@ class BasePlaybookManager:
             for host in hosts:
                 result = cb.host_results.get(host)
                 if state == 'ok':
-                    self.on_host_success(host, result)
+                    self.on_host_success(host, result.get('ok', ''))
                 elif state == 'skipped':
                     pass
                 else:
                     error = hosts.get(host)
-                    self.on_host_error(host, error, result)
+                    self.on_host_error(
+                        host, error,
+                        result.get('failures', '')
+                        or result.get('dark', '')
+                    )
 
     def on_runner_failed(self, runner, e):
         print("Runner failed: {} {}".format(e, self))
 
-    @staticmethod
-    def file_to_json(path):
-        with open(path, 'r') as f:
-            d = json.load(f)
-        return d
-
     @staticmethod
     def json_dumps(data):
         return json.dumps(data, indent=4, sort_keys=True)
 
-    @staticmethod
-    def json_to_file(path, data):
-        with open(path, 'w') as f:
-            json.dump(data, f, indent=4, sort_keys=True)
-
-    def local_gateway_prepare(self, runner):
-        info = self.file_to_json(runner.inventory)
-        servers, not_valid = [], []
-        for k, host in info['all']['hosts'].items():
-            jms_asset, jms_gateway = host.get('jms_asset'), host.get('gateway')
-            if not jms_gateway:
-                continue
-            try:
-                server = SSHTunnelForwarder(
-                    (jms_gateway['address'], jms_gateway['port']),
-                    ssh_username=jms_gateway['username'],
-                    ssh_password=jms_gateway['secret'],
-                    ssh_pkey=jms_gateway['private_key_path'],
-                    remote_bind_address=(jms_asset['address'], jms_asset['port'])
-                )
-                server.start()
-            except Exception as e:
-                err_msg = 'Gateway is not active: %s' % jms_asset.get('name', '')
-                print(f'\033[31m {err_msg} 原因: {e} \033[0m\n')
-                not_valid.append(k)
-            else:
-                host['ansible_host'] = jms_asset['address'] = '127.0.0.1'
-                host['ansible_port'] = jms_asset['port'] = server.local_bind_port
-                servers.append(server)
-
-        # 网域不可连接的，就不继续执行此资源的后续任务了
-        for a in set(not_valid):
-            info['all']['hosts'].pop(a)
-        self.json_to_file(runner.inventory, info)
-        self.gateway_servers[runner.id] = servers
-
-    def local_gateway_clean(self, runner):
-        servers = self.gateway_servers.get(runner.id, [])
-        for s in servers:
-            try:
-                s.stop()
-            except Exception:
-                pass
-
-    def before_runner_start(self, runner):
-        self.local_gateway_prepare(runner)
-
-    def after_runner_end(self, runner):
-        self.local_gateway_clean(runner)
-
     def delete_runtime_dir(self):
         if settings.DEBUG_DEV:
             return
-        shutil.rmtree(self.runtime_dir)
+        shutil.rmtree(self.runtime_dir, ignore_errors=True)
 
     def run(self, *args, **kwargs):
         print(">>> 任务准备阶段\n")
@@ -326,14 +337,16 @@ class BasePlaybookManager:
         for i, runner in enumerate(runners, start=1):
             if len(runners) > 1:
                 print(">>> 开始执行第 {} 批任务".format(i))
-            self.before_runner_start(runner)
+            ssh_tunnel = SSHTunnelManager()
+            ssh_tunnel.local_gateway_prepare(runner)
             try:
+                kwargs.update({"clean_workspace": False})
                 cb = runner.run(**kwargs)
                 self.on_runner_success(runner, cb)
             except Exception as e:
                 self.on_runner_failed(runner, e)
             finally:
-                self.after_runner_end(runner)
+                ssh_tunnel.local_gateway_clean(runner)
                 print('\n')
         self.execution.status = 'success'
         self.execution.date_finished = timezone.now()

apps/assets/automations/gather_facts/database/mysql/main.yml

@@ -2,6 +2,7 @@
   gather_facts: no
   vars:
     ansible_python_interpreter: /opt/py3/bin/python
+    check_ssl: "{{ jms_asset.spec_info.use_ssl and not jms_asset.spec_info.allow_invalid_cert }}"
 
   tasks:
     - name: Get info
@@ -10,10 +11,10 @@
         login_password: "{{ jms_account.secret }}"
         login_host: "{{ jms_asset.address }}"
         login_port: "{{ jms_asset.port }}"
-        check_hostname: "{{ omit if not jms_asset.spec_info.use_ssl else jms_asset.spec_info.allow_invalid_cert }}"
+        check_hostname: "{{ check_ssl if check_ssl else omit }}"
-        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) }}"
+        ca_cert: "{{ jms_asset.secret_info.ca_cert | default(omit) if check_ssl else omit }}"
-        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) }}"
+        client_cert: "{{ jms_asset.secret_info.client_cert | default(omit) if check_ssl else omit }}"
-        client_key: "{{ jms_asset.secret_info.client_key  | default(omit) }}"
+        client_key: "{{ jms_asset.secret_info.client_key | default(omit) if check_ssl else omit }}"
         filter: version
       register: db_info