Merge pull request #5504 from jumpserver/dev

Dev

.github/ISSUE_TEMPLATE/----.md

@@ -2,8 +2,9 @@
 name: 需求建议
 about: 提出针对本项目的想法和建议
 title: "[Feature] "
-labels: 待处理, 需求
-assignees: 'ibuler'
+labels: 类型:需求
+assignees: ibuler
+
 ---
 
 **请描述您的需求或者改进建议.**

.github/ISSUE_TEMPLATE/bug---.md

@@ -2,7 +2,7 @@
 name: Bug 提交
 about: 提交产品缺陷帮助我们更好的改进
 title: "[Bug] "
-labels: bug, 待处理
+labels: 类型:bug
 assignees: wojiushixiaobai
 
 ---

.github/ISSUE_TEMPLATE/question.md

@@ -2,7 +2,7 @@
 name: 问题咨询
 about: 提出针对本项目安装部署、使用及其他方面的相关问题
 title: "[Question] "
-labels: 提问, 待处理
+labels: 类型:提问
 assignees: wojiushixiaobai
 
 ---

Dockerfile

@@ -1,5 +1,6 @@
-FROM registry.fit2cloud.com/public/python:v3 as stage-build
-MAINTAINER Jumpserver Team <ibuler@qq.com>
+# 编译代码
+FROM python:3.8.6-slim as stage-build
+MAINTAINER JumpServer Team <ibuler@qq.com>
 ARG VERSION
 ENV VERSION=$VERSION
 
@@ -8,33 +9,38 @@ ADD . .
 RUN cd utils && bash -ixeu build.sh
 
 
-FROM registry.fit2cloud.com/public/python:v3
+# 构建运行时环境
+FROM python:3.8.6-slim
 ARG PIP_MIRROR=https://pypi.douban.com/simple
 ENV PIP_MIRROR=$PIP_MIRROR
-ARG MYSQL_MIRROR=https://mirrors.tuna.tsinghua.edu.cn/mysql/yum/mysql57-community-el6/
-ENV MYSQL_MIRROR=$MYSQL_MIRROR
+ARG PIP_JMS_MIRROR=https://pypi.douban.com/simple
+ENV PIP_JMS_MIRROR=$PIP_JMS_MIRROR
 
 WORKDIR /opt/jumpserver
 
-COPY ./requirements ./requirements
-RUN useradd jumpserver
-RUN yum -y install epel-release && \
-      echo -e "[mysql]\nname=mysql\nbaseurl=${MYSQL_MIRROR}\ngpgcheck=0\nenabled=1" > /etc/yum.repos.d/mysql.repo
-RUN yum -y install $(cat requirements/rpm_requirements.txt)
-RUN pip install --upgrade pip setuptools==49.6.0 wheel -i ${PIP_MIRROR} && \
-    pip config set global.index-url ${PIP_MIRROR}
-RUN pip install $(grep 'jms' requirements/requirements.txt) -i https://pypi.org/simple
-RUN pip install -r requirements/requirements.txt
+COPY ./requirements/deb_buster_requirements.txt ./requirements/deb_buster_requirements.txt
+RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list \
+    && sed -i 's/security.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list \
+    && apt update \
+    && grep -v '^#' ./requirements/deb_buster_requirements.txt | xargs apt -y install \
+    && localedef -c -f UTF-8 -i zh_CN zh_CN.UTF-8 \
+    && cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
+
+COPY ./requirements/requirements.txt ./requirements/requirements.txt
+RUN pip install --upgrade pip==20.2.4 setuptools==49.6.0 wheel==0.34.2 -i ${PIP_MIRROR} \
+    && pip config set global.index-url ${PIP_MIRROR} \
+    && pip install --no-cache-dir $(grep 'jms' requirements/requirements.txt) -i ${PIP_JMS_MIRROR} \
+    && pip install --no-cache-dir -r requirements/requirements.txt
 
 COPY --from=stage-build /opt/jumpserver/release/jumpserver /opt/jumpserver
-RUN mkdir -p /root/.ssh/ && echo -e "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null" > /root/.ssh/config
+RUN mkdir -p /root/.ssh/ \
+    && echo "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null" > /root/.ssh/config
 
 RUN echo > config.yml
 VOLUME /opt/jumpserver/data
 VOLUME /opt/jumpserver/logs
 
 ENV LANG=zh_CN.UTF-8
-ENV LC_ALL=zh_CN.UTF-8
 
 EXPOSE 8070
 EXPOSE 8080

README.md

@@ -4,9 +4,117 @@
 [![Django](https://img.shields.io/badge/django-2.2-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
 [![Docker Pulls](https://img.shields.io/docker/pulls/jumpserver/jms_all.svg)](https://hub.docker.com/u/jumpserver)
 
-|Developer Wanted|
-|------------------|
-|JumpServer 正在寻找开发者，一起为改变世界做些贡献吧，哪怕一点点，联系我 <ibuler@fit2cloud.com> |
+- [ENGLISH](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
+
+## 紧急BUG修复通知
+JumpServer发现远程执行漏洞，请速度修复
+
+非常感谢  **reactivity of Alibaba Hackerone bug bounty program**(瑞典) 向我们报告了此 BUG
+
+**影响版本:**
+```
+< v2.6.2
+< v2.5.4
+< v2.4.5 
+= v1.5.9
+>= v1.5.3
+```
+**安全版本:**
+```
+>= v2.6.2
+>= v2.5.4
+>= v2.4.5 
+= v1.5.9 （版本号没变）
+< v1.5.3
+```
+
+**修复方案:**
+
+将JumpServer升级至安全版本；
+
+**临时修复方案:**
+
+修改 Nginx 配置文件屏蔽漏洞接口 
+
+```
+/api/v1/authentication/connection-token/
+/api/v1/users/connection-token/
+```
+
+Nginx 配置文件位置
+```
+# 社区老版本
+/etc/nginx/conf.d/jumpserver.conf
+
+# 企业老版本
+jumpserver-release/nginx/http_server.conf
+ 
+# 新版本在 
+jumpserver-release/compose/config_static/http_server.conf
+```
+
+修改 Nginx 配置文件实例
+```
+### 保证在 /api 之前 和 / 之前
+location /api/v1/authentication/connection-token/ {
+   return 403;
+}
+ 
+location /api/v1/users/connection-token/ {
+   return 403;
+}
+### 新增以上这些
+ 
+location /api/ {
+    proxy_set_header X-Real-IP $remote_addr;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_pass http://core:8080;
+  }
+ 
+...
+```
+
+修改完成后重启 nginx
+
+```
+docker方式: 
+docker restart jms_nginx
+
+nginx方式:
+systemctl restart nginx
+
+```
+
+**修复验证**
+
+```
+$ wget https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_bug_check.sh 
+
+# 使用方法 bash jms_bug_check.sh HOST 
+$ bash jms_bug_check.sh demo.jumpserver.org
+漏洞已修复
+```
+
+**入侵检测**
+
+下载脚本到 jumpserver 日志目录，这个目录中存在 gunicorn.log，然后执行
+
+```
+$ pwd
+/opt/jumpserver/core/logs
+
+$ ls gunicorn.log 
+gunicorn.log
+
+$ wget 'https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_check_attack.sh'
+$ bash jms_check_attack.sh
+系统未被入侵
+```
+
+--------------------------
+
+JumpServer 正在寻找开发者，一起为改变世界做些贡献吧，哪怕一点点，联系我 <ibuler@fit2cloud.com> 
 
 JumpServer 是全球首款开源的堡垒机，使用 GNU GPL v2.0 开源协议，是符合 4A 规范的运维安全审计系统。
 
@@ -25,7 +133,8 @@ JumpServer 采纳分布式架构，支持多机房跨区域部署，支持横向
 - 无插件: 仅需浏览器，极致的 Web Terminal 使用体验；
 - 多云支持: 一套系统，同时管理不同云上面的资产；
 - 云端存储: 审计录像云端存储，永不丢失；
-- 多租户: 一套系统，多个子公司和部门同时使用。
+- 多租户: 一套系统，多个子公司和部门同时使用；
+- 多应用支持: 数据库，Windows远程应用，Kubernetes。
 
 ## 版本说明
 
@@ -198,13 +307,61 @@ v2.1.0 是 v2.0.0 之后的功能版本。
     <td>文件传输</td>
     <td>可对文件的上传、下载记录进行审计</td>
   </tr>
+  <tr>
+    <td rowspan="20">数据库审计<br>Database</td>
+    <td rowspan="2">连接方式</td>
+    <td>命令方式</td>
+  </tr>
+  <tr>
+    <td>Web UI方式 (X-PACK)</td>
+  </tr>
+
+  <tr>
+    <td rowspan="4">支持的数据库</td>
+    <td>MySQL</td>
+  </tr>
+  <tr>
+    <td>Oracle (X-PACK)</td>
+  </tr>
+  <tr>
+    <td>MariaDB (X-PACK)</td>
+  </tr>
+  <tr>
+    <td>PostgreSQL (X-PACK)</td>
+  </tr>
+  <tr>
+    <td rowspan="6">功能亮点</td>
+    <td>语法高亮</td>
+  </tr>
+  <tr>
+    <td>SQL格式化</td>
+  </tr>
+  <tr>
+    <td>支持快捷键</td>
+  </tr>
+  <tr>
+    <td>支持选中执行</td>
+  </tr>
+  <tr>
+    <td>SQL历史查询</td>
+  </tr>
+  <tr>
+    <td>支持页面创建 DB, TABLE</td>
+  </tr>
+  <tr>
+    <td rowspan="2">会话审计</td>
+    <td>命令记录</td>
+  </tr>
+  <tr>
+    <td>录像回放</td>
+  </tr>
 </table>
 
 ## 快速开始
 
 - [极速安装](https://docs.jumpserver.org/zh/master/install/setup_by_fast/)
 - [完整文档](https://docs.jumpserver.org)
-- [演示视频](https://jumpserver.oss-cn-hangzhou.aliyuncs.com/jms-media/%E3%80%90%E6%BC%94%E7%A4%BA%E8%A7%86%E9%A2%91%E3%80%91Jumpserver%20%E5%A0%A1%E5%9E%92%E6%9C%BA%20V1.5.0%20%E6%BC%94%E7%A4%BA%E8%A7%86%E9%A2%91%20-%20final.mp4)
+- [演示视频](https://www.bilibili.com/video/BV1ZV41127GB)
 
 ## 组件项目
 - [Lina](https://github.com/jumpserver/lina) JumpServer Web UI 项目
@@ -212,6 +369,11 @@ v2.1.0 是 v2.0.0 之后的功能版本。
 - [Koko](https://github.com/jumpserver/koko) JumpServer 字符协议 Connector 项目，替代原来 Python 版本的 [Coco](https://github.com/jumpserver/coco)
 - [Guacamole](https://github.com/jumpserver/docker-guacamole) JumpServer 图形协议 Connector 项目，依赖 [Apache Guacamole](https://guacamole.apache.org/)
 
+## 致谢
+- [Apache Guacamole](https://guacamole.apache.org/) Web页面连接 RDP, SSH, VNC协议设备，JumpServer 图形化连接依赖
+- [OmniDB](https://omnidb.org/) Web页面连接使用数据库，JumpServer Web数据库依赖
+
+
 ## JumpServer 企业版 
 - [申请企业版试用](https://jinshuju.net/f/kyOYpi)
 > 注：企业版支持离线安装，申请通过后会提供高速下载链接。

README_EN.md

@@ -1,22 +1,135 @@
 ## Jumpserver 
 
-![Total visitor](https://visitor-count-badge.herokuapp.com/total.svg?repo_id=jumpserver)
-![Visitors in today](https://visitor-count-badge.herokuapp.com/today.svg?repo_id=jumpserver)
 [![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
-[![Django](https://img.shields.io/badge/django-2.1-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
-[![Ansible](https://img.shields.io/badge/ansible-2.4.2.0-blue.svg?style=plastic)](https://www.ansible.com/)
-[![Paramiko](https://img.shields.io/badge/paramiko-2.4.1-green.svg?style=plastic)](http://www.paramiko.org/)
+[![Django](https://img.shields.io/badge/django-2.2-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
+[![Docker Pulls](https://img.shields.io/docker/pulls/jumpserver/jms_all.svg)](https://hub.docker.com/u/jumpserver)
 
+---- 
+## CRITICAL BUG WARNING
+
+Recently we have found a critical bug for remote execution vulnerability which leads to pre-auth and info leak, please fix it as soon as possible.
+
+Thanks for **reactivity from Alibaba Hackerone bug bounty program** report us this bug
+
+**Vulnerable version:**
+```
+< v2.6.2
+< v2.5.4
+< v2.4.5 
+= v1.5.9
+>= v1.5.3
+```
+
+**Safe and Stable version:**
+```
+>= v2.6.2
+>= v2.5.4
+>= v2.4.5 
+= v1.5.9 （version tag didn't change）
+< v1.5.3
+```
+
+**Bug Fix Solution:**
+Upgrade to the latest version or the version mentioned above
+
+
+**Temporary Solution (upgrade asap):**
+
+Modify the Nginx config file and disable the vulnerable api listed below
+
+```
+/api/v1/authentication/connection-token/
+/api/v1/users/connection-token/
+```
+
+Path to Nginx config file
+
+```
+# Previous Community version
+/etc/nginx/conf.d/jumpserver.conf
+
+# Previous Enterprise version
+jumpserver-release/nginx/http_server.conf
+ 
+# Latest version
+jumpserver-release/compose/config_static/http_server.conf
+```
+
+Changes in Nginx config file
+
+```
+### Put the following code on top of location server, or before /api and /
+location /api/v1/authentication/connection-token/ {
+   return 403;
+}
+ 
+location /api/v1/users/connection-token/ {
+   return 403;
+}
+### End right here
+ 
+location /api/ {
+    proxy_set_header X-Real-IP $remote_addr;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_pass http://core:8080;
+  }
+ 
+...
+```
+
+Save the file and restart Nginx
+
+```
+docker deployment: 
+$ docker restart jms_nginx
+
+rpm or other deployment:
+$ systemctl restart nginx
+
+```
+
+**Bug Fix Verification**
+
+```
+# Download the following script to check if it is fixed
+$ wget https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_bug_check.sh 
+
+# Run the code to verify it
+$ bash jms_bug_check.sh demo.jumpserver.org
+漏洞已修复 (It means the bug is fixed)
+漏洞未修复 (It means the bug is not fixed and the system is still vulnerable)
+```
+
+
+**Attack Simulation**
+
+Go to the logs directory which should contain gunicorn.log file. Then download the "attack" script and execute it
+
+```
+$ pwd
+/opt/jumpserver/core/logs
+
+$ ls gunicorn.log
+gunicorn.log
+
+$ wget 'https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_check_attack.sh'
+$ bash jms_check_attack.sh
+系统未被入侵 (It means the system is safe)
+系统已被入侵 (It means the system is being attacked)
+```
+
+--------------------------
 
 ----
 
-- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
+- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README.md)
 
-Jumpserver is the first fully open source bastion in the world, based on the GNU GPL v2.0 open source protocol. Jumpserver is a  professional operation and maintenance audit system conforms to 4A specifications.
+Jumpserver is the world's first open-source PAM (Privileged Access Management System) and is licensed under the GNU GPL v2.0. It is a 4A-compliant professional operation and maintenance security audit system.
 
-Jumpserver is developed using Python / Django, conforms to the Web 2.0 specification, and is equipped with the industry-leading Web Terminal solution which have beautiful interface and great user experience.
+Jumpserver uses Python / Django for development, follows Web 2.0 specifications, and is equipped with an industry-leading Web Terminal solution that provides a beautiful user interface and great user experience
 
-Jumpserver adopts a distributed architecture to support multi-branch deployment across multiple areas. The central node provides APIs, and login nodes are deployed in each branch. It can be scaled horizontally without concurrency restrictions.
+Jumpserver adopts a distributed architecture to support multi-branch deployment across multiple cross-regional areas. The central node provides APIs, and login nodes are deployed in each branch. It can be scaled horizontally without concurrency restrictions.
 
 Change the world, starting from little things.
 
@@ -47,7 +160,7 @@ We provide online demo, demo video and screenshots to get you started quickly.
 We provide the SDK for your other systems to quickly interact with the Jumpserver API.
 
 - [Python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver other components use this SDK to complete the interaction.
-- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) 恺珺同学提供的Java版本的SDK thanks to 恺珺 for provide Java SDK
+- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) Thanks to 恺珺 for providing his Java SDK vesrion.
 
 
 ### License & Copyright

apps/.gitattributes

@@ -0,0 +1,2 @@
+*.js linguist-language=python
+*.html linguist-language=python

apps/applications/api/__init__.py

@@ -1,3 +1,3 @@
+from .application import *
+from .mixin import *
 from .remote_app import *
-from .database_app import *
-from .k8s_app import *

apps/applications/api/application.py

@@ -0,0 +1,19 @@
+# coding: utf-8
+#
+
+from orgs.mixins.api import OrgBulkModelViewSet
+
+from ..hands import IsOrgAdminOrAppUser
+from .. import models, serializers
+
+
+__all__ = ['ApplicationViewSet']
+
+
+class ApplicationViewSet(OrgBulkModelViewSet):
+    model = models.Application
+    filterset_fields = ('name', 'type', 'category')
+    search_fields = filterset_fields
+    permission_classes = (IsOrgAdminOrAppUser,)
+    serializer_class = serializers.ApplicationSerializer
+

apps/applications/api/database_app.py

@@ -1,20 +0,0 @@
-# coding: utf-8
-# 
-
-from orgs.mixins.api import OrgBulkModelViewSet
-
-from .. import models
-from .. import serializers
-from ..hands import IsOrgAdminOrAppUser
-
-__all__ = [
-    'DatabaseAppViewSet',
-]
-
-
-class DatabaseAppViewSet(OrgBulkModelViewSet):
-    model = models.DatabaseApp
-    filter_fields = ('name',)
-    search_fields = filter_fields
-    permission_classes = (IsOrgAdminOrAppUser,)
-    serializer_class = serializers.DatabaseAppSerializer

apps/applications/api/k8s_app.py

@@ -1,20 +0,0 @@
-# coding: utf-8
-#
-
-from orgs.mixins.api import OrgBulkModelViewSet
-
-from .. import models
-from .. import serializers
-from ..hands import IsOrgAdminOrAppUser
-
-__all__ = [
-    'K8sAppViewSet',
-]
-
-
-class K8sAppViewSet(OrgBulkModelViewSet):
-    model = models.K8sApp
-    filter_fields = ('name',)
-    search_fields = filter_fields
-    permission_classes = (IsOrgAdminOrAppUser,)
-    serializer_class = serializers.K8sAppSerializer

apps/applications/api/mixin.py

@@ -0,0 +1,89 @@
+from orgs.models import Organization
+
+
+__all__ = ['SerializeApplicationToTreeNodeMixin']
+
+
+class SerializeApplicationToTreeNodeMixin:
+
+    @staticmethod
+    def _serialize_db(db):
+        return {
+            'id': db.id,
+            'name': db.name,
+            'title': db.name,
+            'pId': '',
+            'open': False,
+            'iconSkin': 'database',
+            'meta': {'type': 'database_app'}
+        }
+
+    @staticmethod
+    def _serialize_remote_app(remote_app):
+        return {
+            'id': remote_app.id,
+            'name': remote_app.name,
+            'title': remote_app.name,
+            'pId': '',
+            'open': False,
+            'isParent': False,
+            'iconSkin': 'chrome',
+            'meta': {'type': 'remote_app'}
+        }
+
+    @staticmethod
+    def _serialize_cloud(cloud):
+        return {
+            'id': cloud.id,
+            'name': cloud.name,
+            'title': cloud.name,
+            'pId': '',
+            'open': False,
+            'isParent': False,
+            'iconSkin': 'k8s',
+            'meta': {'type': 'k8s_app'}
+        }
+
+    def _serialize_application(self, application):
+        method_name = f'_serialize_{application.category}'
+        data = getattr(self, method_name)(application)
+        data.update({
+            'pId': application.org.id,
+            'org_name': application.org_name
+        })
+        return data
+
+    def serialize_applications(self, applications):
+        data = [self._serialize_application(application) for application in applications]
+        return data
+
+    @staticmethod
+    def _serialize_organization(org):
+        return {
+            'id': org.id,
+            'name': org.name,
+            'title': org.name,
+            'pId': '',
+            'open': True,
+            'isParent': True,
+            'meta': {
+                'type': 'node'
+            }
+        }
+
+    def serialize_organizations(self, organizations):
+        data = [self._serialize_organization(org) for org in organizations]
+        return data
+
+    @staticmethod
+    def filter_organizations(applications):
+        organizations_id = set(applications.values_list('org_id', flat=True))
+        organizations = [Organization.get_instance(org_id) for org_id in organizations_id]
+        return organizations
+
+    def serialize_applications_with_org(self, applications):
+        organizations = self.filter_organizations(applications)
+        data_organizations = self.serialize_organizations(organizations)
+        data_applications = self.serialize_applications(applications)
+        data = data_organizations + data_applications
+        return data

apps/applications/api/remote_app.py

@@ -1,27 +1,19 @@
 # coding: utf-8
 #
 
-from orgs.mixins.api import OrgBulkModelViewSet
 from orgs.mixins import generics
-from ..hands import IsOrgAdmin, IsAppUser
-from ..models import RemoteApp
-from ..serializers import RemoteAppSerializer, RemoteAppConnectionInfoSerializer
+from ..hands import IsAppUser
+from .. import models
+from ..serializers import RemoteAppConnectionInfoSerializer
+from ..permissions import IsRemoteApp
 
 
 __all__ = [
-    'RemoteAppViewSet', 'RemoteAppConnectionInfoApi',
+    'RemoteAppConnectionInfoApi',
 ]
 
 
-class RemoteAppViewSet(OrgBulkModelViewSet):
-    model = RemoteApp
-    filter_fields = ('name', 'type', 'comment')
-    search_fields = filter_fields
-    permission_classes = (IsOrgAdmin,)
-    serializer_class = RemoteAppSerializer
-
-
 class RemoteAppConnectionInfoApi(generics.RetrieveAPIView):
-    model = RemoteApp
-    permission_classes = (IsAppUser, )
+    model = models.Application
+    permission_classes = (IsAppUser, IsRemoteApp)
     serializer_class = RemoteAppConnectionInfoSerializer

apps/applications/const.py

@@ -1,64 +1,49 @@
 #  coding: utf-8
 #
 
+from django.db.models import TextChoices
 from django.utils.translation import ugettext_lazy as _
 
 
-# RemoteApp
+class ApplicationCategoryChoices(TextChoices):
+    db = 'db', _('Database')
+    remote_app = 'remote_app', _('Remote app')
+    cloud = 'cloud', 'Cloud'
 
-REMOTE_APP_BOOT_PROGRAM_NAME = '||jmservisor'
-
-REMOTE_APP_TYPE_CHROME = 'chrome'
-REMOTE_APP_TYPE_MYSQL_WORKBENCH = 'mysql_workbench'
-REMOTE_APP_TYPE_VMWARE_CLIENT = 'vmware_client'
-REMOTE_APP_TYPE_CUSTOM = 'custom'
-
-# Fields attribute write_only default => False
-
-REMOTE_APP_TYPE_CHROME_FIELDS = [
-    {'name': 'chrome_target'},
-    {'name': 'chrome_username'},
-    {'name': 'chrome_password', 'write_only': True}
-]
-REMOTE_APP_TYPE_MYSQL_WORKBENCH_FIELDS = [
-    {'name': 'mysql_workbench_ip'},
-    {'name': 'mysql_workbench_name'},
-    {'name': 'mysql_workbench_port'},
-    {'name': 'mysql_workbench_username'},
-    {'name': 'mysql_workbench_password', 'write_only': True}
-]
-REMOTE_APP_TYPE_VMWARE_CLIENT_FIELDS = [
-    {'name': 'vmware_target'},
-    {'name': 'vmware_username'},
-    {'name': 'vmware_password', 'write_only': True}
-]
-REMOTE_APP_TYPE_CUSTOM_FIELDS = [
-    {'name': 'custom_cmdline'},
-    {'name': 'custom_target'},
-    {'name': 'custom_username'},
-    {'name': 'custom_password', 'write_only': True}
-]
-
-REMOTE_APP_TYPE_FIELDS_MAP = {
-    REMOTE_APP_TYPE_CHROME: REMOTE_APP_TYPE_CHROME_FIELDS,
-    REMOTE_APP_TYPE_MYSQL_WORKBENCH: REMOTE_APP_TYPE_MYSQL_WORKBENCH_FIELDS,
-    REMOTE_APP_TYPE_VMWARE_CLIENT: REMOTE_APP_TYPE_VMWARE_CLIENT_FIELDS,
-    REMOTE_APP_TYPE_CUSTOM: REMOTE_APP_TYPE_CUSTOM_FIELDS
-}
-
-REMOTE_APP_TYPE_CHOICES = (
-    (REMOTE_APP_TYPE_CHROME, 'Chrome'),
-    (REMOTE_APP_TYPE_MYSQL_WORKBENCH, 'MySQL Workbench'),
-    (REMOTE_APP_TYPE_VMWARE_CLIENT, 'vSphere Client'),
-    (REMOTE_APP_TYPE_CUSTOM, _('Custom')),
-)
+    @classmethod
+    def get_label(cls, category):
+        return dict(cls.choices).get(category, '')
 
 
-# DatabaseApp
+class ApplicationTypeChoices(TextChoices):
+    # db category
+    mysql = 'mysql', 'MySQL'
+    oracle = 'oracle', 'Oracle'
+    pgsql = 'postgresql', 'PostgreSQL'
+    mariadb = 'mariadb', 'MariaDB'
 
+    # remote-app category
+    chrome = 'chrome', 'Chrome'
+    mysql_workbench = 'mysql_workbench', 'MySQL Workbench'
+    vmware_client = 'vmware_client', 'vSphere Client'
+    custom = 'custom', _('Custom')
 
-DATABASE_APP_TYPE_MYSQL = 'mysql'
+    # cloud category
+    k8s = 'k8s', 'Kubernetes'
+
+    @classmethod
+    def get_label(cls, tp):
+        return dict(cls.choices).get(tp, '')
+
+    @classmethod
+    def db_types(cls):
+        return [cls.mysql.value, cls.oracle.value, cls.pgsql.value, cls.mariadb.value]
+
+    @classmethod
+    def remote_app_types(cls):
+        return [cls.chrome.value, cls.mysql_workbench.value, cls.vmware_client.value, cls.custom.value]
+
+    @classmethod
+    def cloud_types(cls):
+        return [cls.k8s.value]
 
-DATABASE_APP_TYPE_CHOICES = (
-    (DATABASE_APP_TYPE_MYSQL, 'MySQL'),
-)

apps/applications/migrations/0006_application.py

@@ -0,0 +1,140 @@
+# Generated by Django 2.2.13 on 2020-10-19 12:01
+
+from django.db import migrations, models
+import django.db.models.deletion
+import django_mysql.models
+import uuid
+
+
+CATEGORY_DB_LIST = ['mysql', 'oracle', 'postgresql', 'mariadb']
+CATEGORY_REMOTE_LIST = ['chrome', 'mysql_workbench', 'vmware_client', 'custom']
+CATEGORY_CLOUD_LIST = ['k8s']
+
+CATEGORY_DB = 'db'
+CATEGORY_REMOTE = 'remote_app'
+CATEGORY_CLOUD = 'cloud'
+CATEGORY_LIST = [CATEGORY_DB, CATEGORY_REMOTE, CATEGORY_CLOUD]
+
+
+def get_application_category(old_app):
+    _type = old_app.type
+    if _type in CATEGORY_DB_LIST:
+        category = CATEGORY_DB
+    elif _type in CATEGORY_REMOTE_LIST:
+        category = CATEGORY_REMOTE
+    elif _type in CATEGORY_CLOUD_LIST:
+        category = CATEGORY_CLOUD
+    else:
+        category = None
+    return category
+
+
+def common_to_application_json(old_app):
+    category = get_application_category(old_app)
+    date_updated = old_app.date_updated if hasattr(old_app, 'date_updated') else old_app.date_created
+    return {
+        'id': old_app.id,
+        'name': old_app.name,
+        'type': old_app.type,
+        'category': category,
+        'comment': old_app.comment,
+        'created_by': old_app.created_by,
+        'date_created': old_app.date_created,
+        'date_updated': date_updated,
+        'org_id': old_app.org_id
+    }
+
+
+def db_to_application_json(database):
+    app_json = common_to_application_json(database)
+    app_json.update({
+        'attrs': {
+            'host': database.host,
+            'port': database.port,
+            'database': database.database
+        }
+    })
+    return app_json
+
+
+def remote_to_application_json(remote):
+    app_json = common_to_application_json(remote)
+    attrs = {
+        'asset': str(remote.asset.id),
+        'path': remote.path,
+    }
+    attrs.update(remote.params)
+    app_json.update({
+        'attrs': attrs
+    })
+    return app_json
+
+
+def k8s_to_application_json(k8s):
+    app_json = common_to_application_json(k8s)
+    app_json.update({
+        'attrs': {
+            'cluster': k8s.cluster
+        }
+    })
+    return app_json
+
+
+def migrate_and_integrate_applications(apps, schema_editor):
+    db_alias = schema_editor.connection.alias
+
+    database_app_model = apps.get_model("applications", "DatabaseApp")
+    remote_app_model = apps.get_model("applications", "RemoteApp")
+    k8s_app_model = apps.get_model("applications", "K8sApp")
+
+    database_apps = database_app_model.objects.using(db_alias).all()
+    remote_apps = remote_app_model.objects.using(db_alias).all()
+    k8s_apps = k8s_app_model.objects.using(db_alias).all()
+
+    database_applications = [db_to_application_json(db_app) for db_app in database_apps]
+    remote_applications = [remote_to_application_json(remote_app) for remote_app in remote_apps]
+    k8s_applications = [k8s_to_application_json(k8s_app) for k8s_app in k8s_apps]
+
+    applications_json = database_applications + remote_applications + k8s_applications
+    application_model = apps.get_model("applications", "Application")
+    applications = [
+        application_model(**application_json)
+        for application_json in applications_json
+        if application_json['category'] in CATEGORY_LIST
+    ]
+    for application in applications:
+        if application_model.objects.using(db_alias).filter(name=application.name).exists():
+            application.name = '{}-{}'.format(application.name, application.type)
+        application.save()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0057_fill_node_value_assets_amount_and_parent_key'),
+        ('applications', '0005_k8sapp'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Application',
+            fields=[
+                ('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('created_by', models.CharField(blank=True, max_length=32, null=True, verbose_name='Created by')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+                ('date_updated', models.DateTimeField(auto_now=True, verbose_name='Date updated')),
+                ('name', models.CharField(max_length=128, verbose_name='Name')),
+                ('category', models.CharField(choices=[('db', 'Database'), ('remote_app', 'Remote app'), ('cloud', 'Cloud')], max_length=16, verbose_name='Category')),
+                ('type', models.CharField(choices=[('mysql', 'MySQL'), ('oracle', 'Oracle'), ('postgresql', 'PostgreSQL'), ('mariadb', 'MariaDB'), ('chrome', 'Chrome'), ('mysql_workbench', 'MySQL Workbench'), ('vmware_client', 'vSphere Client'), ('custom', 'Custom'), ('k8s', 'Kubernetes')], max_length=16, verbose_name='Type')),
+                ('attrs', django_mysql.models.JSONField(default=dict)),
+                ('comment', models.TextField(blank=True, default='', max_length=128, verbose_name='Comment')),
+                ('domain', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='applications', to='assets.Domain', verbose_name='Domain')),
+            ],
+            options={
+                'ordering': ('name',),
+                'unique_together': {('org_id', 'name')},
+            },
+        ),
+        migrations.RunPython(migrate_and_integrate_applications),
+    ]

apps/applications/migrations/0007_auto_20201119_1110.py

@@ -0,0 +1,18 @@
+# Generated by Django 3.1 on 2020-11-19 03:10
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('applications', '0006_application'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='application',
+            name='attrs',
+            field=models.JSONField(),
+        ),
+    ]

apps/applications/migrations/0008_auto_20210104_0435.py

@@ -0,0 +1,28 @@
+# Generated by Django 3.1 on 2021-01-03 20:35
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('perms', '0017_auto_20210104_0435'),
+        ('applications', '0007_auto_20201119_1110'),
+    ]
+
+    operations = [
+        migrations.DeleteModel(
+            name='DatabaseApp',
+        ),
+        migrations.DeleteModel(
+            name='K8sApp',
+        ),
+        migrations.AlterField(
+            model_name='application',
+            name='attrs',
+            field=models.JSONField(default=dict, verbose_name='Attrs'),
+        ),
+        migrations.DeleteModel(
+            name='RemoteApp',
+        ),
+    ]

apps/applications/models/__init__.py

@@ -1,3 +1 @@
-from .remote_app import *
-from .database_app import *
-from .k8s_app import *
+from .application import *

apps/applications/models/application.py

@@ -0,0 +1,37 @@
+from django.db import models
+from django.utils.translation import ugettext_lazy as _
+
+from orgs.mixins.models import OrgModelMixin
+from common.mixins import CommonModelMixin
+from .. import const
+
+
+class Application(CommonModelMixin, OrgModelMixin):
+    name = models.CharField(max_length=128, verbose_name=_('Name'))
+    category = models.CharField(
+        max_length=16, choices=const.ApplicationCategoryChoices.choices, verbose_name=_('Category')
+    )
+    type = models.CharField(
+        max_length=16, choices=const.ApplicationTypeChoices.choices, verbose_name=_('Type')
+    )
+    domain = models.ForeignKey(
+        'assets.Domain', null=True, blank=True, related_name='applications',
+        on_delete=models.SET_NULL, verbose_name=_("Domain"),
+    )
+    attrs = models.JSONField(default=dict, verbose_name=_('Attrs'))
+    comment = models.TextField(
+        max_length=128, default='', blank=True, verbose_name=_('Comment')
+    )
+
+    class Meta:
+        unique_together = [('org_id', 'name')]
+        ordering = ('name',)
+
+    def __str__(self):
+        category_display = self.get_category_display()
+        type_display = self.get_type_display()
+        return f'{self.name}({type_display})[{category_display}]'
+
+    @property
+    def category_remote_app(self):
+        return self.category == const.ApplicationCategoryChoices.remote_app.value

apps/applications/models/database_app.py

@@ -1,42 +0,0 @@
-# coding: utf-8
-#
-
-import uuid
-from django.db import models
-from django.utils.translation import ugettext_lazy as _
-
-from orgs.mixins.models import OrgModelMixin
-from common.mixins import CommonModelMixin
-from .. import const
-
-
-__all__ = ['DatabaseApp']
-
-
-class DatabaseApp(CommonModelMixin, OrgModelMixin):
-    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
-    name = models.CharField(max_length=128, verbose_name=_('Name'))
-    type = models.CharField(
-        default=const.DATABASE_APP_TYPE_MYSQL,
-        choices=const.DATABASE_APP_TYPE_CHOICES,
-        max_length=128, verbose_name=_('Type')
-    )
-    host = models.CharField(
-        max_length=128, verbose_name=_('Host'), db_index=True
-    )
-    port = models.IntegerField(default=3306, verbose_name=_('Port'))
-    database = models.CharField(
-        max_length=128, blank=True, null=True, verbose_name=_('Database'),
-        db_index=True
-    )
-    comment = models.TextField(
-        max_length=128, default='', blank=True, verbose_name=_('Comment')
-    )
-
-    def __str__(self):
-        return self.name
-
-    class Meta:
-        unique_together = [('org_id', 'name'), ]
-        verbose_name = _("DatabaseApp")
-        ordering = ('name', )

apps/applications/models/k8s_app.py

@@ -1,27 +0,0 @@
-from django.utils.translation import gettext_lazy as _
-
-from common.db import models
-from orgs.mixins.models import OrgModelMixin
-
-
-class K8sApp(OrgModelMixin, models.JMSModel):
-    class TYPE(models.ChoiceSet):
-        K8S = 'k8s', _('Kubernetes')
-
-    name = models.CharField(max_length=128, verbose_name=_('Name'))
-    type = models.CharField(
-        default=TYPE.K8S, choices=TYPE.choices,
-        max_length=128, verbose_name=_('Type')
-    )
-    cluster = models.CharField(max_length=1024, verbose_name=_('Cluster'))
-    comment = models.TextField(
-        max_length=128, default='', blank=True, verbose_name=_('Comment')
-    )
-
-    def __str__(self):
-        return self.name
-
-    class Meta:
-        unique_together = [('org_id', 'name'), ]
-        verbose_name = _('KubernetesApp')
-        ordering = ('name', )

apps/applications/models/remote_app.py

@@ -1,78 +0,0 @@
-# coding: utf-8
-#
-
-import uuid
-from django.db import models
-from django.utils.translation import ugettext_lazy as _
-
-from orgs.mixins.models import OrgModelMixin
-from common.fields.model import EncryptJsonDictTextField
-
-from .. import const
-
-
-__all__ = [
-    'RemoteApp',
-]
-
-
-class RemoteApp(OrgModelMixin):
-    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
-    name = models.CharField(max_length=128, verbose_name=_('Name'))
-    asset = models.ForeignKey(
-        'assets.Asset', on_delete=models.CASCADE, verbose_name=_('Asset')
-    )
-    type = models.CharField(
-        default=const.REMOTE_APP_TYPE_CHROME,
-        choices=const.REMOTE_APP_TYPE_CHOICES,
-        max_length=128, verbose_name=_('App type')
-    )
-    path = models.CharField(
-        max_length=128, blank=False, null=False,
-        verbose_name=_('App path')
-    )
-    params = EncryptJsonDictTextField(
-        max_length=4096, default={}, blank=True, null=True,
-        verbose_name=_('Parameters')
-    )
-    created_by = models.CharField(
-        max_length=32, null=True, blank=True, verbose_name=_('Created by')
-    )
-    date_created = models.DateTimeField(
-        auto_now_add=True, null=True, blank=True, verbose_name=_('Date created')
-    )
-    comment = models.TextField(
-        max_length=128, default='', blank=True, verbose_name=_('Comment')
-    )
-
-    class Meta:
-        verbose_name = _("RemoteApp")
-        unique_together = [('org_id', 'name')]
-        ordering = ('name', )
-
-    def __str__(self):
-        return self.name
-
-    @property
-    def parameters(self):
-        """
-        返回Guacamole需要的RemoteApp配置参数信息中的parameters参数
-        """
-        _parameters = list()
-        _parameters.append(self.type)
-        path = '\"%s\"' % self.path
-        _parameters.append(path)
-        for field in const.REMOTE_APP_TYPE_FIELDS_MAP[self.type]:
-            value = self.params.get(field['name'])
-            if value is None:
-                continue
-            _parameters.append(value)
-        _parameters = ' '.join(_parameters)
-        return _parameters
-
-    @property
-    def asset_info(self):
-        return {
-            'id': self.asset.id,
-            'hostname': self.asset.hostname
-        }

apps/applications/permissions.py

@@ -0,0 +1,9 @@
+from rest_framework import permissions
+
+
+__all__ = ['IsRemoteApp']
+
+
+class IsRemoteApp(permissions.BasePermission):
+    def has_object_permission(self, request, view, obj):
+        return obj.category_remote_app

apps/applications/serializers/__init__.py

@@ -1,3 +1,2 @@
+from .application import *
 from .remote_app import *
-from .database_app import *
-from .k8s_app import *

apps/applications/serializers/application.py

@@ -0,0 +1,64 @@
+# coding: utf-8
+#
+
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+from orgs.mixins.serializers import BulkOrgResourceModelSerializer
+from common.drf.serializers import MethodSerializer
+from .attrs import category_serializer_classes_mapping, type_serializer_classes_mapping
+
+from .. import models
+
+__all__ = [
+    'ApplicationSerializer', 'ApplicationSerializerMixin',
+]
+
+
+class ApplicationSerializerMixin(serializers.Serializer):
+    attrs = MethodSerializer()
+
+    def get_attrs_serializer(self):
+        default_serializer = serializers.Serializer(read_only=True)
+        if isinstance(self.instance, models.Application):
+            _type = self.instance.type
+            _category = self.instance.category
+        else:
+            _type = self.context['request'].query_params.get('type')
+            _category = self.context['request'].query_params.get('category')
+
+        if _type:
+            serializer_class = type_serializer_classes_mapping.get(_type)
+        elif _category:
+            serializer_class = category_serializer_classes_mapping.get(_category)
+        else:
+            serializer_class = default_serializer
+
+        if not serializer_class:
+            serializer_class = default_serializer
+
+        if isinstance(serializer_class, type):
+            serializer = serializer_class()
+        else:
+            serializer = serializer_class
+        return serializer
+
+
+class ApplicationSerializer(ApplicationSerializerMixin, BulkOrgResourceModelSerializer):
+    category_display = serializers.ReadOnlyField(source='get_category_display', label=_('Category'))
+    type_display = serializers.ReadOnlyField(source='get_type_display', label=_('Type'))
+
+    class Meta:
+        model = models.Application
+        fields = [
+            'id', 'name', 'category', 'category_display', 'type', 'type_display', 'attrs',
+            'domain', 'created_by', 'date_created', 'date_updated', 'comment'
+        ]
+        read_only_fields = [
+            'created_by', 'date_created', 'date_updated', 'get_type_display',
+        ]
+
+    def validate_attrs(self, attrs):
+        _attrs = self.instance.attrs if self.instance else {}
+        _attrs.update(attrs)
+        return _attrs
+

apps/applications/serializers/attrs/__init__.py

@@ -0,0 +1 @@
+from .attrs import *

apps/applications/serializers/attrs/application_category/__init__.py

@@ -0,0 +1,3 @@
+from .remote_app import *
+from .db import *
+from .cloud import *

apps/applications/serializers/attrs/application_category/cloud.py

@@ -0,0 +1,9 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+
+__all__ = ['CloudSerializer']
+
+
+class CloudSerializer(serializers.Serializer):
+    cluster = serializers.CharField(max_length=1024, label=_('Cluster'), allow_null=True)

apps/applications/serializers/attrs/application_category/db.py

@@ -0,0 +1,15 @@
+# coding: utf-8
+#
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+
+__all__ = ['DBSerializer']
+
+
+class DBSerializer(serializers.Serializer):
+    host = serializers.CharField(max_length=128, label=_('Host'), allow_null=True)
+    port = serializers.IntegerField(label=_('Port'), allow_null=True)
+    database = serializers.CharField(
+        max_length=128, required=True, allow_null=True, label=_('Database')
+    )

apps/applications/serializers/attrs/application_category/remote_app.py

@@ -0,0 +1,52 @@
+# coding: utf-8
+#
+
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+from django.core.exceptions import ObjectDoesNotExist
+
+from common.utils import get_logger, is_uuid
+from assets.models import Asset
+
+logger = get_logger(__file__)
+
+
+__all__ = ['RemoteAppSerializer']
+
+
+class CharPrimaryKeyRelatedField(serializers.PrimaryKeyRelatedField):
+
+    def to_internal_value(self, data):
+        instance = super().to_internal_value(data)
+        return str(instance.id)
+
+    def to_representation(self, value):
+        # value is instance.id
+        if self.pk_field is not None:
+            return self.pk_field.to_representation(value)
+        return value
+
+
+class RemoteAppSerializer(serializers.Serializer):
+    asset_info = serializers.SerializerMethodField()
+    asset = CharPrimaryKeyRelatedField(
+        queryset=Asset.objects, required=False, label=_("Asset"), allow_null=True
+    )
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), allow_null=True
+    )
+
+    @staticmethod
+    def get_asset_info(obj):
+        asset_id = obj.get('asset')
+        if not asset_id or is_uuid(asset_id):
+            return {}
+        try:
+            asset = Asset.objects.filter(id=str(asset_id)).values_list('id', 'hostname')
+        except ObjectDoesNotExist as e:
+            logger.error(e)
+            return {}
+        if not asset:
+            return {}
+        asset_info = {'id': str(asset[0]), 'hostname': asset[1]}
+        return asset_info

apps/applications/serializers/attrs/application_type/__init__.py

@@ -0,0 +1,12 @@
+
+from .mysql import *
+from .mariadb import *
+from .oracle import *
+from .pgsql import *
+
+from .chrome import *
+from .mysql_workbench import *
+from .vmware_client import *
+from .custom import *
+
+from .k8s import *

apps/applications/serializers/attrs/application_type/chrome.py

@@ -0,0 +1,26 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['ChromeSerializer']
+
+
+class ChromeSerializer(RemoteAppSerializer):
+    CHROME_PATH = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'
+
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), default=CHROME_PATH, allow_null=True,
+    )
+    chrome_target = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Target URL'), allow_null=True,
+    )
+    chrome_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'), allow_null=True,
+    )
+    chrome_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True
+    )
+

apps/applications/serializers/attrs/application_type/custom.py

@@ -0,0 +1,27 @@
+
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['CustomSerializer']
+
+
+class CustomSerializer(RemoteAppSerializer):
+    custom_cmdline = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Operating parameter'),
+        allow_null=True,
+    )
+    custom_target = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Target url'),
+        allow_null=True,
+    )
+    custom_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'),
+        allow_null=True,
+    )
+    custom_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True,
+    )

apps/applications/serializers/attrs/application_type/k8s.py

@@ -0,0 +1,8 @@
+from ..application_category import CloudSerializer
+
+
+__all__ = ['K8SSerializer']
+
+
+class K8SSerializer(CloudSerializer):
+    pass

apps/applications/serializers/attrs/application_type/mariadb.py

@@ -0,0 +1,8 @@
+from .mysql import MySQLSerializer
+
+
+__all__ = ['MariaDBSerializer']
+
+
+class MariaDBSerializer(MySQLSerializer):
+    pass

apps/applications/serializers/attrs/application_type/mysql.py

@@ -0,0 +1,15 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+from ..application_category import DBSerializer
+
+
+__all__ = ['MySQLSerializer']
+
+
+class MySQLSerializer(DBSerializer):
+    port = serializers.IntegerField(default=3306, label=_('Port'), allow_null=True)
+
+
+
+

apps/applications/serializers/attrs/application_type/mysql_workbench.py

@@ -0,0 +1,36 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['MySQLWorkbenchSerializer']
+
+
+class MySQLWorkbenchSerializer(RemoteAppSerializer):
+    MYSQL_WORKBENCH_PATH = 'C:\Program Files\MySQL\MySQL Workbench 8.0 CE\MySQLWorkbench.exe'
+
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), default=MYSQL_WORKBENCH_PATH,
+        allow_null=True,
+    )
+    mysql_workbench_ip = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('IP'),
+        allow_null=True,
+    )
+    mysql_workbench_port = serializers.IntegerField(
+        required=False, label=_('Port'),
+        allow_null=True,
+    )
+    mysql_workbench_name = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Database'),
+        allow_null=True,
+    )
+    mysql_workbench_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'),
+        allow_null=True,
+    )
+    mysql_workbench_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True,
+    )

apps/applications/serializers/attrs/application_type/oracle.py

@@ -0,0 +1,12 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+from ..application_category import DBSerializer
+
+
+__all__ = ['OracleSerializer']
+
+
+class OracleSerializer(DBSerializer):
+    port = serializers.IntegerField(default=1521, label=_('Port'), allow_null=True)
+

apps/applications/serializers/attrs/application_type/pgsql.py

@@ -0,0 +1,12 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+from ..application_category import DBSerializer
+
+
+__all__ = ['PostgreSerializer']
+
+
+class PostgreSerializer(DBSerializer):
+    port = serializers.IntegerField(default=5432, label=_('Port'), allow_null=True)
+

apps/applications/serializers/attrs/application_type/vmware_client.py

@@ -0,0 +1,32 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['VMwareClientSerializer']
+
+
+class VMwareClientSerializer(RemoteAppSerializer):
+    PATH = r'''
+    C:\Program Files (x86)\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\VpxClient
+    .exe
+    '''
+    VMWARE_CLIENT_PATH = ''.join(PATH.split())
+
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), default=VMWARE_CLIENT_PATH,
+        allow_null=True
+    )
+    vmware_target = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Target URL'),
+        allow_null=True
+    )
+    vmware_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'),
+        allow_null=True
+    )
+    vmware_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True
+    )

apps/applications/serializers/attrs/attrs.py

@@ -0,0 +1,42 @@
+from rest_framework import serializers
+from applications import const
+from . import application_category, application_type
+
+
+__all__ = [
+    'category_serializer_classes_mapping',
+    'type_serializer_classes_mapping',
+    'get_serializer_class_by_application_type',
+]
+
+
+# define `attrs` field `category serializers mapping`
+# ---------------------------------------------------
+
+category_serializer_classes_mapping = {
+    const.ApplicationCategoryChoices.db.value: application_category.DBSerializer,
+    const.ApplicationCategoryChoices.remote_app.value: application_category.RemoteAppSerializer,
+    const.ApplicationCategoryChoices.cloud.value: application_category.CloudSerializer,
+}
+
+# define `attrs` field `type serializers mapping`
+# -----------------------------------------------
+
+type_serializer_classes_mapping = {
+    # db
+    const.ApplicationTypeChoices.mysql.value: application_type.MySQLSerializer,
+    const.ApplicationTypeChoices.mariadb.value: application_type.MariaDBSerializer,
+    const.ApplicationTypeChoices.oracle.value: application_type.OracleSerializer,
+    const.ApplicationTypeChoices.pgsql.value: application_type.PostgreSerializer,
+    # remote-app
+    const.ApplicationTypeChoices.chrome.value: application_type.ChromeSerializer,
+    const.ApplicationTypeChoices.mysql_workbench.value: application_type.MySQLWorkbenchSerializer,
+    const.ApplicationTypeChoices.vmware_client.value: application_type.VMwareClientSerializer,
+    const.ApplicationTypeChoices.custom.value: application_type.CustomSerializer,
+    # cloud
+    const.ApplicationTypeChoices.k8s.value: application_type.K8SSerializer
+}
+
+
+def get_serializer_class_by_application_type(_application_type):
+    return type_serializer_classes_mapping.get(_application_type)

apps/applications/serializers/database_app.py

@@ -1,26 +0,0 @@
-# coding: utf-8
-#
-
-from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-from common.serializers import AdaptedBulkListSerializer
-
-from .. import models
-
-__all__ = [
-    'DatabaseAppSerializer',
-]
-
-
-class DatabaseAppSerializer(BulkOrgResourceModelSerializer):
-
-    class Meta:
-        model = models.DatabaseApp
-        list_serializer_class = AdaptedBulkListSerializer
-        fields = [
-            'id', 'name', 'type', 'get_type_display', 'host', 'port',
-            'database', 'comment', 'created_by', 'date_created', 'date_updated',
-        ]
-        read_only_fields = [
-            'created_by', 'date_created', 'date_updated'
-            'get_type_display',
-        ]

apps/applications/serializers/k8s_app.py

@@ -1,22 +0,0 @@
-from rest_framework import serializers
-
-from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-from .. import models
-
-__all__ = [
-    'K8sAppSerializer',
-]
-
-
-class K8sAppSerializer(BulkOrgResourceModelSerializer):
-    type_display = serializers.CharField(source='get_type_display', read_only=True)
-
-    class Meta:
-        model = models.K8sApp
-        fields = [
-            'id', 'name', 'type', 'type_display', 'comment', 'created_by',
-            'date_created', 'date_updated', 'cluster'
-        ]
-        read_only_fields = [
-            'id', 'created_by', 'date_created', 'date_updated',
-        ]

apps/applications/serializers/remote_app.py

@@ -1,86 +1,57 @@
 # coding: utf-8
 #
-
-import copy
 from rest_framework import serializers
-
-from common.serializers import AdaptedBulkListSerializer
-from common.fields.serializer import CustomMetaDictField
-from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-
-from .. import const
-from ..models import RemoteApp
+from common.utils import get_logger
+from ..models import Application
 
 
-__all__ = [
-    'RemoteAppSerializer', 'RemoteAppConnectionInfoSerializer',
-]
+logger = get_logger(__file__)
 
 
-class RemoteAppParamsDictField(CustomMetaDictField):
-    type_fields_map = const.REMOTE_APP_TYPE_FIELDS_MAP
-    default_type = const.REMOTE_APP_TYPE_CHROME
-    convert_key_remove_type_prefix = False
-    convert_key_to_upper = False
-
-
-class RemoteAppSerializer(BulkOrgResourceModelSerializer):
-    params = RemoteAppParamsDictField()
-    type_fields_map = const.REMOTE_APP_TYPE_FIELDS_MAP
-
-    class Meta:
-        model = RemoteApp
-        list_serializer_class = AdaptedBulkListSerializer
-        fields = [
-            'id', 'name', 'asset', 'asset_info', 'type', 'get_type_display',
-            'path', 'params', 'date_created', 'created_by', 'comment',
-        ]
-        read_only_fields = [
-            'created_by', 'date_created', 'asset_info',
-            'get_type_display'
-        ]
-
-    def process_params(self, instance, validated_data):
-        new_params = copy.deepcopy(validated_data.get('params', {}))
-        tp = validated_data.get('type', '')
-
-        if tp != instance.type:
-            return new_params
-
-        old_params = instance.params
-        fields = self.type_fields_map.get(instance.type, [])
-        for field in fields:
-            if not field.get('write_only', False):
-                continue
-            field_name = field['name']
-            new_value = new_params.get(field_name, '')
-            old_value = old_params.get(field_name, '')
-            field_value = new_value if new_value else old_value
-            new_params[field_name] = field_value
-
-        return new_params
-
-    def update(self, instance, validated_data):
-        params = self.process_params(instance, validated_data)
-        validated_data['params'] = params
-        return super().update(instance, validated_data)
+__all__ = ['RemoteAppConnectionInfoSerializer']
 
 
 class RemoteAppConnectionInfoSerializer(serializers.ModelSerializer):
     parameter_remote_app = serializers.SerializerMethodField()
+    asset = serializers.SerializerMethodField()
 
     class Meta:
-        model = RemoteApp
+        model = Application
         fields = [
             'id', 'name', 'asset', 'parameter_remote_app',
         ]
         read_only_fields = ['parameter_remote_app']
 
     @staticmethod
-    def get_parameter_remote_app(obj):
-        parameter = {
-            'program': const.REMOTE_APP_BOOT_PROGRAM_NAME,
+    def get_asset(obj):
+        return obj.attrs.get('asset')
+
+    @staticmethod
+    def get_parameters(obj):
+        """
+        返回Guacamole需要的RemoteApp配置参数信息中的parameters参数
+        """
+        from .attrs import get_serializer_class_by_application_type
+        serializer_class = get_serializer_class_by_application_type(obj.type)
+        fields = serializer_class().get_fields()
+
+        parameters = [obj.type]
+        for field_name in list(fields.keys()):
+            if field_name in ['asset']:
+                continue
+            value = obj.attrs.get(field_name)
+            if not value:
+                continue
+            if field_name == 'path':
+                value = '\"%s\"' % value
+            parameters.append(str(value))
+
+        parameters = ' '.join(parameters)
+        return parameters
+
+    def get_parameter_remote_app(self, obj):
+        return {
+            'program': '||jmservisor',
             'working_directory': '',
-            'parameters': obj.parameters,
+            'parameters': self.get_parameters(obj)
         }
-        return parameter

apps/applications/urls/api_urls.py

@@ -1,25 +1,20 @@
 # coding:utf-8
 #
-
-from django.urls import path, re_path
+from django.urls import path
 from rest_framework_bulk.routes import BulkRouter
-
-from common import api as capi
 from .. import api
 
+
 app_name = 'applications'
 
+
 router = BulkRouter()
-router.register(r'remote-apps', api.RemoteAppViewSet, 'remote-app')
-router.register(r'database-apps', api.DatabaseAppViewSet, 'database-app')
-router.register(r'k8s-apps', api.K8sAppViewSet, 'k8s-app')
+router.register(r'applications', api.ApplicationViewSet, 'application')
+
 
 urlpatterns = [
     path('remote-apps/<uuid:pk>/connection-info/', api.RemoteAppConnectionInfoApi.as_view(), name='remote-app-connection-info'),
 ]
 
-old_version_urlpatterns = [
-    re_path('(?P<resource>remote-app)/.*', capi.redirect_plural_name_api)
-]
 
-urlpatterns += router.urls + old_version_urlpatterns
+urlpatterns += router.urls

apps/applications/urls/views_urls.py

@@ -1,7 +0,0 @@
-# coding:utf-8
-from django.urls import path
-
-app_name = 'applications'
-
-urlpatterns = [
-]

apps/assets/api/admin_user.py

@@ -29,8 +29,8 @@ class AdminUserViewSet(OrgBulkModelViewSet):
     Admin user api set, for add,delete,update,list,retrieve resource
     """
     model = AdminUser
-    filter_fields = ("name", "username")
-    search_fields = filter_fields
+    filterset_fields = ("name", "username")
+    search_fields = filterset_fields
     serializer_class = serializers.AdminUserSerializer
     permission_classes = (IsOrgAdmin,)
 
@@ -93,9 +93,8 @@ class AdminUserTestConnectiveApi(generics.RetrieveAPIView):
 class AdminUserAssetsListView(generics.ListAPIView):
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.AssetSimpleSerializer
-    filter_fields = ("hostname", "ip")
-    http_method_names = ['get']
-    search_fields = filter_fields
+    filterset_fields = ("hostname", "ip")
+    search_fields = filterset_fields
 
     def get_object(self):
         pk = self.kwargs.get('pk')

apps/assets/api/asset.py

@@ -3,6 +3,8 @@
 from assets.api import FilterAssetByNodeMixin
 from rest_framework.viewsets import ModelViewSet
 from rest_framework.generics import RetrieveAPIView
+from rest_framework.response import Response
+from rest_framework import status
 from django.shortcuts import get_object_or_404
 
 from common.utils import get_logger, get_object_or_none
@@ -12,7 +14,7 @@ from orgs.mixins import generics
 from ..models import Asset, Node, Platform
 from .. import serializers
 from ..tasks import (
-    update_asset_hardware_info_manual, test_asset_connectivity_manual
+    update_assets_hardware_info_manual, test_assets_connectivity_manual
 )
 from ..filters import FilterAssetByNodeFilterBackend, LabelFilterBackend, IpInFilterBackend
 
@@ -21,7 +23,7 @@ logger = get_logger(__file__)
 __all__ = [
     'AssetViewSet', 'AssetPlatformRetrieveApi',
     'AssetGatewayListApi', 'AssetPlatformViewSet',
-    'AssetTaskCreateApi',
+    'AssetTaskCreateApi', 'AssetsTaskCreateApi',
 ]
 
 
@@ -30,10 +32,15 @@ class AssetViewSet(FilterAssetByNodeMixin, OrgBulkModelViewSet):
     API endpoint that allows Asset to be viewed or edited.
     """
     model = Asset
-    filter_fields = (
-        "hostname", "ip", "systemuser__id", "admin_user__id", "platform__base",
-        "is_active", 'ip'
-    )
+    filterset_fields = {
+        'hostname': ['exact'],
+        'ip': ['exact'],
+        'systemuser__id': ['exact'],
+        'admin_user__id': ['exact'],
+        'platform__base': ['exact'],
+        'is_active': ['exact'],
+        'protocols': ['exact', 'icontains']
+    }
     search_fields = ("hostname", "ip")
     ordering_fields = ("hostname", "ip", "port", "cpu_cores")
     serializer_classes = {
@@ -74,7 +81,7 @@ class AssetPlatformViewSet(ModelViewSet):
     queryset = Platform.objects.all()
     permission_classes = (IsSuperUser,)
     serializer_class = serializers.PlatformSerializer
-    filter_fields = ['name', 'base']
+    filterset_fields = ['name', 'base']
     search_fields = ['name']
 
     def get_permissions(self):
@@ -90,32 +97,43 @@ class AssetPlatformViewSet(ModelViewSet):
         return super().check_object_permissions(request, obj)
 
 
-class AssetTaskCreateApi(generics.CreateAPIView):
+class AssetsTaskMixin:
+    def perform_assets_task(self, serializer):
+        data = serializer.validated_data
+        assets = data['assets']
+        action = data['action']
+        if action == "refresh":
+            task = update_assets_hardware_info_manual.delay(assets)
+        else:
+            task = test_assets_connectivity_manual.delay(assets)
+        data = getattr(serializer, '_data', {})
+        data["task"] = task.id
+        setattr(serializer, '_data', data)
+
+    def perform_create(self, serializer):
+        self.perform_assets_task(serializer)
+
+
+class AssetTaskCreateApi(AssetsTaskMixin, generics.CreateAPIView):
     model = Asset
     serializer_class = serializers.AssetTaskSerializer
     permission_classes = (IsOrgAdmin,)
 
-    def get_object(self):
-        pk = self.kwargs.get("pk")
-        instance = get_object_or_404(Asset, pk=pk)
-        return instance
+    def create(self, request, *args, **kwargs):
+        pk = self.kwargs.get('pk')
+        request.data['assets'] = [pk]
+        return super().create(request, *args, **kwargs)
 
-    def perform_create(self, serializer):
-        asset = self.get_object()
-        action = serializer.validated_data["action"]
-        if action == "refresh":
-            task = update_asset_hardware_info_manual.delay(asset)
-        else:
-            task = test_asset_connectivity_manual.delay(asset)
-        data = getattr(serializer, '_data', {})
-        data["task"] = task.id
-        setattr(serializer, '_data', data)
+
+class AssetsTaskCreateApi(AssetsTaskMixin, generics.CreateAPIView):
+    model = Asset
+    serializer_class = serializers.AssetTaskSerializer
+    permission_classes = (IsOrgAdmin,)
 
 
 class AssetGatewayListApi(generics.ListAPIView):
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.GatewayWithAuthSerializer
-    model = Asset
 
     def get_queryset(self):
         asset_id = self.kwargs.get('pk')

apps/assets/api/asset_user.py

@@ -28,7 +28,7 @@ logger = get_logger(__name__)
 class AssetUserFilterBackend(filters.BaseFilterBackend):
     def filter_queryset(self, request, queryset, view):
         kwargs = {}
-        for field in view.filter_fields:
+        for field in view.filterset_fields:
             value = request.GET.get(field)
             if not value:
                 continue
@@ -78,7 +78,7 @@ class AssetUserViewSet(CommonApiMixin, BulkModelViewSet):
         'retrieve': serializers.AssetUserReadSerializer,
     }
     permission_classes = [IsOrgAdminOrAppUser]
-    filter_fields = [
+    filterset_fields = [
         "id", "ip", "hostname", "username",
         "asset_id", "node_id",
         "prefer", "prefer_id",
@@ -131,7 +131,7 @@ class AssetUserTaskCreateAPI(generics.CreateAPIView):
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.AssetUserTaskSerializer
     filter_backends = AssetUserViewSet.filter_backends
-    filter_fields = AssetUserViewSet.filter_fields
+    filterset_fields = AssetUserViewSet.filterset_fields
 
     def get_asset_users(self):
         manager = AssetUserManager()

apps/assets/api/cmd_filter.py

@@ -14,16 +14,16 @@ __all__ = ['CommandFilterViewSet', 'CommandFilterRuleViewSet']
 
 class CommandFilterViewSet(OrgBulkModelViewSet):
     model = CommandFilter
-    filter_fields = ("name",)
-    search_fields = filter_fields
+    filterset_fields = ("name",)
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.CommandFilterSerializer
 
 
 class CommandFilterRuleViewSet(OrgBulkModelViewSet):
     model = CommandFilterRule
-    filter_fields = ("content",)
-    search_fields = filter_fields
+    filterset_fields = ("content",)
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.CommandFilterRuleSerializer
 

apps/assets/api/domain.py

@@ -1,7 +1,9 @@
 # ~*~ coding: utf-8 ~*~
 
-from rest_framework.views import APIView, Response
 from django.views.generic.detail import SingleObjectMixin
+from django.utils.translation import ugettext as _
+from rest_framework.views import APIView, Response
+from rest_framework.serializers import ValidationError
 
 from common.utils import get_logger
 from common.permissions import IsOrgAdmin, IsOrgAdminOrAppUser
@@ -16,8 +18,8 @@ __all__ = ['DomainViewSet', 'GatewayViewSet', "GatewayTestConnectionApi"]
 
 class DomainViewSet(OrgBulkModelViewSet):
     model = Domain
-    filter_fields = ("name", )
-    search_fields = filter_fields
+    filterset_fields = ("name", )
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.DomainSerializer
 
@@ -29,7 +31,7 @@ class DomainViewSet(OrgBulkModelViewSet):
 
 class GatewayViewSet(OrgBulkModelViewSet):
     model = Gateway
-    filter_fields = ("domain__name", "name", "username", "ip", "domain")
+    filterset_fields = ("domain__name", "name", "username", "ip", "domain")
     search_fields = ("domain__name", "name", "username", "ip")
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.GatewaySerializer
@@ -42,6 +44,10 @@ class GatewayTestConnectionApi(SingleObjectMixin, APIView):
     def post(self, request, *args, **kwargs):
         self.object = self.get_object(Gateway.objects.all())
         local_port = self.request.data.get('port') or self.object.port
+        try:
+            local_port = int(local_port)
+        except ValueError:
+            raise ValidationError({'port': _('Number required')})
         ok, e = self.object.test_connective(local_port=local_port)
         if ok:
             return Response("ok")

apps/assets/api/favorite_asset.py

@@ -13,7 +13,7 @@ __all__ = ['FavoriteAssetViewSet']
 class FavoriteAssetViewSet(BulkModelViewSet):
     serializer_class = FavoriteAssetSerializer
     permission_classes = (IsValidUser,)
-    filter_fields = ['asset']
+    filterset_fields = ['asset']
 
     def dispatch(self, request, *args, **kwargs):
         with tmp_to_root_org():

apps/assets/api/gathered_user.py

@@ -18,5 +18,5 @@ class GatheredUserViewSet(OrgModelViewSet):
     permission_classes = [IsOrgAdmin]
     extra_filter_backends = [AssetRelatedByNodeFilterBackend]
 
-    filter_fields = ['asset', 'username', 'present', 'asset__ip', 'asset__hostname', 'asset_id']
+    filterset_fields = ['asset', 'username', 'present', 'asset__ip', 'asset__hostname', 'asset_id']
     search_fields = ['username', 'asset__ip', 'asset__hostname']

apps/assets/api/label.py

@@ -28,8 +28,8 @@ __all__ = ['LabelViewSet']
 
 class LabelViewSet(OrgBulkModelViewSet):
     model = Label
-    filter_fields = ("name", "value")
-    search_fields = filter_fields
+    filterset_fields = ("name", "value")
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.LabelSerializer
 

apps/assets/api/mixin.py

@@ -69,6 +69,7 @@ class SerializeToTreeNodeMixin:
                         'ip': asset.ip,
                         'protocols': asset.protocols_as_list,
                         'platform': asset.platform_base,
+                        'org_name': asset.org_name
                     },
                 }
             }

apps/assets/api/node.py

@@ -5,11 +5,13 @@ from collections import namedtuple, defaultdict
 from rest_framework import status
 from rest_framework.serializers import ValidationError
 from rest_framework.response import Response
+from rest_framework.decorators import action
 from django.utils.translation import ugettext_lazy as _
 from django.shortcuts import get_object_or_404, Http404
 from django.utils.decorators import method_decorator
 from django.db.models.signals import m2m_changed
 
+from common.const.http import POST
 from common.exceptions import SomeoneIsDoingThis
 from common.const.signals import PRE_REMOVE, POST_REMOVE
 from assets.models import Asset
@@ -19,6 +21,8 @@ from common.const.distributed_lock_key import UPDATE_NODE_TREE_LOCK_KEY
 from orgs.mixins.api import OrgModelViewSet
 from orgs.mixins import generics
 from orgs.lock import org_level_transaction_lock
+from orgs.utils import current_org
+from assets.tasks import check_node_assets_amount_task
 from ..hands import IsOrgAdmin
 from ..models import Node
 from ..tasks import (
@@ -41,11 +45,16 @@ __all__ = [
 
 class NodeViewSet(OrgModelViewSet):
     model = Node
-    filter_fields = ('value', 'key', 'id')
+    filterset_fields = ('value', 'key', 'id')
     search_fields = ('value', )
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.NodeSerializer
 
+    @action(methods=[POST], detail=False, url_name='launch-check-assets-amount-task')
+    def launch_check_assets_amount_task(self, request):
+        task = check_node_assets_amount_task.delay(current_org.id)
+        return Response(data={'task': task.id})
+
     # 仅支持根节点指直接创建，子节点下的节点需要通过children接口创建
     def perform_create(self, serializer):
         child_key = Node.org_root().get_next_child_key()
@@ -61,6 +70,9 @@ class NodeViewSet(OrgModelViewSet):
 
     def destroy(self, request, *args, **kwargs):
         node = self.get_object()
+        if node.is_org_root():
+            error = _("You can't delete the root node ({})".format(node.value))
+            return Response(data={'error': error}, status=status.HTTP_403_FORBIDDEN)
         if node.has_children_or_has_assets():
             error = _("Deletion failed and the node contains children or assets")
             return Response(data={'error': error}, status=status.HTTP_403_FORBIDDEN)
@@ -173,7 +185,7 @@ class NodeChildrenAsTreeApi(SerializeToTreeNodeMixin, NodeChildrenApi):
             return []
         assets = self.instance.get_assets().only(
             "id", "hostname", "ip", "os",
-            "org_id", "protocols",
+            "org_id", "protocols", "is_active"
         )
         return self.serialize_assets(assets, self.instance.key)
 
@@ -201,10 +213,8 @@ class NodeAddChildrenApi(generics.UpdateAPIView):
     def put(self, request, *args, **kwargs):
         instance = self.get_object()
         nodes_id = request.data.get("nodes")
-        children = [get_object_or_none(Node, id=pk) for pk in nodes_id]
+        children = Node.objects.filter(id__in=nodes_id)
         for node in children:
-            if not node:
-                continue
             node.parent = instance
         return Response("OK")
 

apps/assets/api/system_user.py

@@ -3,7 +3,8 @@ from django.shortcuts import get_object_or_404
 from rest_framework.response import Response
 
 from common.utils import get_logger
-from common.permissions import IsOrgAdmin, IsOrgAdminOrAppUser, IsAppUser
+from common.permissions import IsOrgAdmin, IsOrgAdminOrAppUser
+from common.drf.filters import CustomFilter
 from orgs.mixins.api import OrgBulkModelViewSet
 from orgs.mixins import generics
 from orgs.utils import tmp_to_org
@@ -12,14 +13,14 @@ from .. import serializers
 from ..serializers import SystemUserWithAuthInfoSerializer
 from ..tasks import (
     push_system_user_to_assets_manual, test_system_user_connectivity_manual,
-    push_system_user_a_asset_manual,
+    push_system_user_to_assets
 )
 
 
 logger = get_logger(__file__)
 __all__ = [
     'SystemUserViewSet', 'SystemUserAuthInfoApi', 'SystemUserAssetAuthInfoApi',
-    'SystemUserCommandFilterRuleListApi', 'SystemUserTaskApi',
+    'SystemUserCommandFilterRuleListApi', 'SystemUserTaskApi', 'SystemUserAssetsListView',
 ]
 
 
@@ -28,8 +29,12 @@ class SystemUserViewSet(OrgBulkModelViewSet):
     System user api set, for add,delete,update,list,retrieve resource
     """
     model = SystemUser
-    filter_fields = ("name", "username", "protocol")
-    search_fields = filter_fields
+    filterset_fields = {
+        'name': ['exact'],
+        'username': ['exact'],
+        'protocol': ['exact', 'in']
+    }
+    search_fields = filterset_fields
     serializer_class = serializers.SystemUserSerializer
     serializer_classes = {
         'default': serializers.SystemUserSerializer,
@@ -82,18 +87,18 @@ class SystemUserTaskApi(generics.CreateAPIView):
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.SystemUserTaskSerializer
 
-    def do_push(self, system_user, asset=None):
-        if asset is None:
+    def do_push(self, system_user, assets_id=None):
+        if assets_id is None:
             task = push_system_user_to_assets_manual.delay(system_user)
         else:
             username = self.request.query_params.get('username')
-            task = push_system_user_a_asset_manual.delay(
-                system_user, asset, username=username
+            task = push_system_user_to_assets.delay(
+                system_user.id, assets_id, username=username
             )
         return task
 
     @staticmethod
-    def do_test(system_user, asset=None):
+    def do_test(system_user):
         task = test_system_user_connectivity_manual.delay(system_user)
         return task
 
@@ -104,11 +109,16 @@ class SystemUserTaskApi(generics.CreateAPIView):
     def perform_create(self, serializer):
         action = serializer.validated_data["action"]
         asset = serializer.validated_data.get('asset')
+        assets = serializer.validated_data.get('assets') or []
+
         system_user = self.get_object()
         if action == 'push':
-            task = self.do_push(system_user, asset)
+            assets = [asset] if asset else assets
+            assets_id = [asset.id for asset in assets]
+            assets_id = assets_id if assets_id else None
+            task = self.do_push(system_user, assets_id)
         else:
-            task = self.do_test(system_user, asset)
+            task = self.do_test(system_user)
         data = getattr(serializer, '_data', {})
         data["task"] = task.id
         setattr(serializer, '_data', data)
@@ -125,3 +135,18 @@ class SystemUserCommandFilterRuleListApi(generics.ListAPIView):
         pk = self.kwargs.get('pk', None)
         system_user = get_object_or_404(SystemUser, pk=pk)
         return system_user.cmd_filter_rules
+
+
+class SystemUserAssetsListView(generics.ListAPIView):
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.AssetSimpleSerializer
+    filterset_fields = ("hostname", "ip")
+    search_fields = filterset_fields
+
+    def get_object(self):
+        pk = self.kwargs.get('pk')
+        return get_object_or_404(SystemUser, pk=pk)
+
+    def get_queryset(self):
+        system_user = self.get_object()
+        return system_user.get_all_assets()

apps/assets/api/system_user_relation.py

@@ -65,7 +65,7 @@ class SystemUserAssetRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserAssetRelationSerializer
     model = models.SystemUser.assets.through
     permission_classes = (IsOrgAdmin,)
-    filter_fields = [
+    filterset_fields = [
         'id', 'asset', 'systemuser',
     ]
     search_fields = [
@@ -91,11 +91,11 @@ class SystemUserNodeRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserNodeRelationSerializer
     model = models.SystemUser.nodes.through
     permission_classes = (IsOrgAdmin,)
-    filter_fields = [
+    filterset_fields = [
         'id', 'node', 'systemuser',
     ]
     search_fields = [
-        "node__value", "systemuser__name", "systemuser_username"
+        "node__value", "systemuser__name", "systemuser__username"
     ]
 
     def get_objects_attr(self):
@@ -112,7 +112,7 @@ class SystemUserUserRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserUserRelationSerializer
     model = models.SystemUser.users.through
     permission_classes = (IsOrgAdmin,)
-    filter_fields = [
+    filterset_fields = [
         'id', 'user', 'systemuser',
     ]
     search_fields = [

apps/assets/migrations/0058_auto_20201023_1115.py

@@ -0,0 +1,27 @@
+# Generated by Django 2.2.13 on 2020-10-23 03:15
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0057_fill_node_value_assets_amount_and_parent_key'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='asset',
+            options={'ordering': ['-date_created'], 'verbose_name': 'Asset'},
+        ),
+        migrations.AlterField(
+            model_name='asset',
+            name='comment',
+            field=models.TextField(blank=True, default='', verbose_name='Comment'),
+        ),
+        migrations.AlterField(
+            model_name='commandfilterrule',
+            name='content',
+            field=models.TextField(help_text='One line one command', verbose_name='Content'),
+        ),
+    ]

apps/assets/migrations/0059_auto_20201027_1905.py

@@ -0,0 +1,28 @@
+# Generated by Django 2.2.13 on 2020-10-27 11:05
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0058_auto_20201023_1115'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='systemuser',
+            name='protocol',
+            field=models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp'), ('telnet', 'telnet'), ('vnc', 'vnc'), ('mysql', 'mysql'), ('oracle', 'oracle'), ('mariadb', 'mariadb'), ('postgresql', 'postgresql'), ('k8s', 'k8s')], default='ssh', max_length=16, verbose_name='Protocol'),
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='ad_domain',
+            field=models.CharField(default='', max_length=256),
+        ),
+        migrations.AlterField(
+            model_name='gateway',
+            name='ip',
+            field=models.CharField(db_index=True, max_length=128, verbose_name='IP'),
+        ),
+    ]

apps/assets/migrations/0060_node_full_value.py

@@ -0,0 +1,58 @@
+# Generated by Django 2.2.13 on 2020-10-26 11:31
+
+from django.db import migrations, models
+
+
+def get_node_ancestor_keys(key, with_self=False):
+    parent_keys = []
+    key_list = key.split(":")
+    if not with_self:
+        key_list.pop()
+    for i in range(len(key_list)):
+        parent_keys.append(":".join(key_list))
+        key_list.pop()
+    return parent_keys
+
+
+def migrate_nodes_value_with_slash(apps, schema_editor):
+    model = apps.get_model("assets", "Node")
+    db_alias = schema_editor.connection.alias
+    nodes = model.objects.using(db_alias).filter(value__contains='/')
+    print('')
+    print("- Start migrate node value if has /")
+    for i, node in enumerate(list(nodes)):
+        new_value = node.value.replace('/', '|')
+        print("{} start migrate node value: {} => {}".format(i, node.value, new_value))
+        node.value = new_value
+        node.save()
+
+
+def migrate_nodes_full_value(apps, schema_editor):
+    model = apps.get_model("assets", "Node")
+    db_alias = schema_editor.connection.alias
+    nodes = model.objects.using(db_alias).all()
+    print("- Start migrate node full value")
+    for i, node in enumerate(list(nodes)):
+        print("{} start migrate {} node full value".format(i, node.value))
+        ancestor_keys = get_node_ancestor_keys(node.key, True)
+        values = model.objects.filter(key__in=ancestor_keys).values_list('key', 'value')
+        values = [v for k, v in sorted(values, key=lambda x: len(x[0]))]
+        node.full_value = '/' + '/'.join(values)
+        node.save()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0059_auto_20201027_1905'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='node',
+            name='full_value',
+            field=models.CharField(default='', max_length=4096, verbose_name='Full value'),
+        ),
+        migrations.RunPython(migrate_nodes_value_with_slash),
+        migrations.RunPython(migrate_nodes_full_value)
+    ]

apps/assets/migrations/0061_auto_20201116_1757.py

@@ -0,0 +1,17 @@
+# Generated by Django 2.2.13 on 2020-11-16 09:57
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0060_node_full_value'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='node',
+            options={'ordering': ['value'], 'verbose_name': 'Node'},
+        ),
+    ]

apps/assets/migrations/0062_auto_20201117_1938.py

@@ -0,0 +1,17 @@
+# Generated by Django 2.2.13 on 2020-11-17 11:38
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0061_auto_20201116_1757'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='asset',
+            options={'ordering': ['hostname', 'ip'], 'verbose_name': 'Asset'},
+        ),
+    ]

apps/assets/migrations/0063_migrate_default_node_key.py

@@ -0,0 +1,72 @@
+# Generated by Jiangjie.Bai on 2020-12-01 10:47
+
+from django.db import migrations
+from django.db.models import Q
+
+default_node_value = 'Default'  # Always
+old_default_node_key = '0'  # Version <= 1.4.3
+new_default_node_key = '1'  # Version >= 1.4.4
+
+
+def compute_parent_key(key):
+    try:
+        return key[:key.rindex(':')]
+    except ValueError:
+        return ''
+
+
+def migrate_default_node_key(apps, schema_editor):
+    """ 将已经存在的Default节点的key从0修改为1 """
+    # 1.4.3版本中Default节点的key为0
+    print('')
+    Node = apps.get_model('assets', 'Node')
+    Asset = apps.get_model('assets', 'Asset')
+
+    # key为0的节点
+    old_default_node = Node.objects.filter(key=old_default_node_key, value=default_node_value).first()
+    if not old_default_node:
+        print(f'Check old default node `key={old_default_node_key} value={default_node_value}` not exists')
+        return
+    print(f'Check old default node `key={old_default_node_key} value={default_node_value}` exists')
+    # key为1的节点
+    new_default_node = Node.objects.filter(key=new_default_node_key, value=default_node_value).first()
+    if new_default_node:
+        print(f'Check new default node `key={new_default_node_key} value={default_node_value}` exists')
+        all_assets = Asset.objects.filter(
+            Q(nodes__key__startswith=f'{new_default_node_key}:') | Q(nodes__key=new_default_node_key)
+        ).distinct()
+        if all_assets:
+            print(f'Check new default node has assets (count: {len(all_assets)})')
+            return
+        all_children = Node.objects.filter(key__startswith=f'{new_default_node_key}:')
+        if all_children:
+            print(f'Check new default node has children nodes (count: {len(all_children)})')
+            return
+        print(f'Check new default node not has assets and children nodes, delete it.')
+        new_default_node.delete()
+    # 执行修改
+    print(f'Modify old default node `key` from `{old_default_node_key}` to `{new_default_node_key}`')
+    nodes = Node.objects.filter(
+        Q(key__istartswith=f'{old_default_node_key}:') | Q(key=old_default_node_key)
+    )
+    for node in nodes:
+        old_key = node.key
+        key_list = old_key.split(':', maxsplit=1)
+        key_list[0] = new_default_node_key
+        new_key = ':'.join(key_list)
+        node.key = new_key
+        node.parent_key = compute_parent_key(node.key)
+    # 批量更新
+    print(f'Bulk update nodes `key` and `parent_key`, (count: {len(nodes)})')
+    Node.objects.bulk_update(nodes, ['key', 'parent_key'])
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0062_auto_20201117_1938'),
+    ]
+
+    operations = [
+        migrations.RunPython(migrate_default_node_key)
+    ]

apps/assets/migrations/0064_auto_20201203_1100.py

@@ -0,0 +1,17 @@
+# Generated by Django 3.1 on 2020-12-03 03:00
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0063_migrate_default_node_key'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='node',
+            options={'ordering': ['parent_key', 'value'], 'verbose_name': 'Node'},
+        ),
+    ]

apps/assets/migrations/0065_auto_20210121_1549.py

@@ -0,0 +1,17 @@
+# Generated by Django 3.1 on 2021-01-21 07:49
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0064_auto_20201203_1100'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='domain',
+            options={'ordering': ('name',), 'verbose_name': 'Domain'},
+        ),
+    ]

apps/assets/models/asset.py

@@ -227,7 +227,7 @@ class Asset(ProtocolsMixin, NodesRelationMixin, OrgModelMixin):
     labels = models.ManyToManyField('assets.Label', blank=True, related_name='assets', verbose_name=_("Labels"))
     created_by = models.CharField(max_length=128, null=True, blank=True, verbose_name=_('Created by'))
     date_created = models.DateTimeField(auto_now_add=True, null=True, blank=True, verbose_name=_('Date created'))
-    comment = models.TextField(max_length=128, default='', blank=True, verbose_name=_('Comment'))
+    comment = models.TextField(default='', blank=True, verbose_name=_('Comment'))
 
     objects = AssetManager.from_queryset(AssetQuerySet)()
     org_objects = AssetOrgManager.from_queryset(AssetQuerySet)()
@@ -313,6 +313,12 @@ class Asset(ProtocolsMixin, NodesRelationMixin, OrgModelMixin):
         }
         return info
 
+    def nodes_display(self):
+        names = []
+        for n in self.nodes.all():
+            names.append(n.full_value)
+        return names
+
     def as_node(self):
         from .node import Node
         fake_node = Node()
@@ -355,3 +361,4 @@ class Asset(ProtocolsMixin, NodesRelationMixin, OrgModelMixin):
     class Meta:
         unique_together = [('org_id', 'hostname')]
         verbose_name = _("Asset")
+        ordering = ["hostname", "ip"]

apps/assets/models/authbook.py

@@ -57,7 +57,7 @@ class AuthBook(BaseUser):
         同时设置自己的 is_latest=True, version=max_version + 1
         """
         username = kwargs['username']
-        asset = kwargs['asset']
+        asset = kwargs.get('asset') or kwargs.get('asset_id')
         with transaction.atomic():
             # 使用select_for_update限制并发创建相同的username、asset条目
             instances = cls.objects.select_for_update().filter(username=username, asset=asset)

apps/assets/models/base.py

@@ -158,9 +158,11 @@ class AuthMixin:
         if update_fields:
             self.save(update_fields=update_fields)
 
-    def has_special_auth(self, asset=None):
+    def has_special_auth(self, asset=None, username=None):
         from .authbook import AuthBook
-        queryset = AuthBook.objects.filter(username=self.username)
+        if username is None:
+            username = self.username
+        queryset = AuthBook.objects.filter(username=username)
         if asset:
             queryset = queryset.filter(asset=asset)
         return queryset.exists()

apps/assets/models/cmd_filter.py

@@ -52,7 +52,7 @@ class CommandFilterRule(OrgModelMixin):
     type = models.CharField(max_length=16, default=TYPE_COMMAND, choices=TYPE_CHOICES, verbose_name=_("Type"))
     priority = models.IntegerField(default=50, verbose_name=_("Priority"), help_text=_("1-100, the higher will be match first"),
                                    validators=[MinValueValidator(1), MaxValueValidator(100)])
-    content = models.TextField(max_length=1024, verbose_name=_("Content"), help_text=_("One line one command"))
+    content = models.TextField(verbose_name=_("Content"), help_text=_("One line one command"))
     action = models.IntegerField(default=ACTION_DENY, choices=ACTION_CHOICES, verbose_name=_("Action"))
     comment = models.CharField(max_length=64, blank=True, default='', verbose_name=_("Comment"))
     date_created = models.DateTimeField(auto_now_add=True)

apps/assets/models/domain.py

@@ -9,6 +9,7 @@ import paramiko
 from django.db import models
 from django.utils.translation import ugettext_lazy as _
 
+from common.utils.strings import no_special_chars
 from orgs.mixins.models import OrgModelMixin
 from .base import BaseUser
 
@@ -25,6 +26,7 @@ class Domain(OrgModelMixin):
     class Meta:
         verbose_name = _("Domain")
         unique_together = [('org_id', 'name')]
+        ordering = ('name',)
 
     def __str__(self):
         return self.name
@@ -47,7 +49,7 @@ class Gateway(BaseUser):
         (PROTOCOL_SSH, 'ssh'),
         (PROTOCOL_RDP, 'rdp'),
     )
-    ip = models.GenericIPAddressField(max_length=32, verbose_name=_('IP'), db_index=True)
+    ip = models.CharField(max_length=128, verbose_name=_('IP'), db_index=True)
     port = models.IntegerField(default=22, verbose_name=_('Port'))
     protocol = models.CharField(choices=PROTOCOL_CHOICES, max_length=16, default=PROTOCOL_SSH, verbose_name=_("Protocol"))
     domain = models.ForeignKey(Domain, on_delete=models.CASCADE, verbose_name=_("Domain"))
@@ -64,8 +66,8 @@ class Gateway(BaseUser):
     def test_connective(self, local_port=None):
         if local_port is None:
             local_port = self.port
-        if self.password and not re.match(r'\w+$', self.password):
-            return False, _("Password should not contain special characters")
+        if self.password and not no_special_chars(self.password):
+            return False, _("Password should not contains special characters")
 
         client = paramiko.SSHClient()
         client.set_missing_host_key_policy(paramiko.AutoAddPolicy())

apps/assets/models/favorite_asset.py

@@ -20,9 +20,13 @@ class FavoriteAsset(CommonModelMixin):
         return cls.objects.filter(user=user).values_list('asset', flat=True)
 
     @classmethod
-    def get_user_favorite_assets(cls, user):
+    def get_user_favorite_assets(cls, user, asset_perms_id=None):
         from assets.models import Asset
-        from perms.utils.user_asset_permission import get_user_granted_all_assets
-        asset_ids = get_user_granted_all_assets(user).values_list('id', flat=True)
+        from perms.utils.asset.user_permission import get_user_granted_all_assets
+        asset_ids = get_user_granted_all_assets(
+            user,
+            via_mapping_node=False,
+            asset_perms_id=asset_perms_id
+        ).values_list('id', flat=True)
         query_name = cls.asset.field.related_query_name()
         return Asset.org_objects.filter(**{f'{query_name}__user_id': user.id}, id__in=asset_ids).distinct()

apps/assets/models/node.py

@@ -13,7 +13,7 @@ from django.db.transaction import atomic
 from common.utils import get_logger
 from common.utils.common import lazyproperty
 from orgs.mixins.models import OrgModelMixin, OrgManager
-from orgs.utils import get_current_org, tmp_to_org, current_org
+from orgs.utils import get_current_org, tmp_to_org
 from orgs.models import Organization
 
 
@@ -38,6 +38,7 @@ class FamilyMixin:
     __children = None
     __all_children = None
     is_node = True
+    child_mark: int
 
     @staticmethod
     def clean_children_keys(nodes_keys):
@@ -103,7 +104,7 @@ class FamilyMixin:
             if value is None:
                 value = child_key
             child = self.__class__.objects.create(
-                id=_id, key=child_key, value=value, parent_key=self.key,
+                id=_id, key=child_key, value=value
             )
             return child
 
@@ -121,11 +122,22 @@ class FamilyMixin:
             created = True
         return child, created
 
+    def get_valid_child_mark(self):
+        key = "{}:{}".format(self.key, self.child_mark)
+        if not self.__class__.objects.filter(key=key).exists():
+            return self.child_mark
+        children_keys = self.get_children().values_list('key', flat=True)
+        children_keys_last = [key.split(':')[-1] for key in children_keys]
+        children_keys_last = [int(k) for k in children_keys_last if k.strip().isdigit()]
+        max_key_last = max(children_keys_last) if children_keys_last else 1
+        return max_key_last + 1
+
     def get_next_child_key(self):
-        mark = self.child_mark
-        self.child_mark += 1
+        child_mark = self.get_valid_child_mark()
+        key = "{}:{}".format(self.key, child_mark)
+        self.child_mark = child_mark + 1
         self.save()
-        return "{}:{}".format(self.key, mark)
+        return key
 
     def get_next_child_preset_name(self):
         name = ugettext("New node")
@@ -205,6 +217,30 @@ class FamilyMixin:
             sibling = sibling.exclude(key=self.key)
         return sibling
 
+    @classmethod
+    def create_node_by_full_value(cls, full_value):
+        if not full_value:
+            return []
+        nodes_family = full_value.split('/')
+        nodes_family = [v for v in nodes_family if v]
+        org_root = cls.org_root()
+        if nodes_family[0] == org_root.value:
+            nodes_family = nodes_family[1:]
+        return cls.create_nodes_recurse(nodes_family, org_root)
+
+    @classmethod
+    def create_nodes_recurse(cls, values, parent=None):
+        values = [v for v in values if v]
+        if not values:
+            return None
+        if parent is None:
+            parent = cls.org_root()
+        value = values[0]
+        child, created = parent.get_or_create_child(value=value)
+        if len(values) == 1:
+            return child
+        return cls.create_nodes_recurse(values[1:], child)
+
     def get_family(self):
         ancestors = self.get_ancestors()
         children = self.get_all_children()
@@ -328,7 +364,10 @@ class SomeNodesMixin:
 
     @classmethod
     def org_root(cls):
-        root = cls.objects.filter(parent_key='').exclude(key__startswith='-')
+        root = cls.objects.filter(parent_key='')\
+            .filter(key__regex=r'^[0-9]+$')\
+            .exclude(key__startswith='-')\
+            .order_by('key')
         if root:
             return root[0]
         else:
@@ -372,6 +411,7 @@ class Node(OrgModelMixin, SomeNodesMixin, FamilyMixin, NodeAssetsMixin):
     id = models.UUIDField(default=uuid.uuid4, primary_key=True)
     key = models.CharField(unique=True, max_length=64, verbose_name=_("Key"))  # '1:1:1:1'
     value = models.CharField(max_length=128, verbose_name=_("Value"))
+    full_value = models.CharField(max_length=4096, verbose_name=_('Full value'), default='')
     child_mark = models.IntegerField(default=0)
     date_create = models.DateTimeField(auto_now_add=True)
     parent_key = models.CharField(max_length=64, verbose_name=_("Parent key"),
@@ -384,10 +424,10 @@ class Node(OrgModelMixin, SomeNodesMixin, FamilyMixin, NodeAssetsMixin):
 
     class Meta:
         verbose_name = _("Node")
-        ordering = ['key']
+        ordering = ['parent_key', 'value']
 
     def __str__(self):
-        return self.value
+        return self.full_value
 
     # def __eq__(self, other):
     #     if not other:
@@ -411,15 +451,14 @@ class Node(OrgModelMixin, SomeNodesMixin, FamilyMixin, NodeAssetsMixin):
     def name(self):
         return self.value
 
-    @lazyproperty
-    def full_value(self):
+    def computed_full_value(self):
         # 不要在列表中调用该属性
         values = self.__class__.objects.filter(
             key__in=self.get_ancestor_keys()
         ).values_list('key', 'value')
         values = [v for k, v in sorted(values, key=lambda x: len(x[0]))]
-        values.append(self.value)
-        return ' / '.join(values)
+        values.append(str(self.value))
+        return '/' + '/'.join(values)
 
     @property
     def level(self):
@@ -458,3 +497,27 @@ class Node(OrgModelMixin, SomeNodesMixin, FamilyMixin, NodeAssetsMixin):
         if self.has_children_or_has_assets():
             return
         return super().delete(using=using, keep_parents=keep_parents)
+
+    def update_child_full_value(self):
+        nodes = self.get_all_children(with_self=True)
+        sort_key_func = lambda n: [int(i) for i in n.key.split(':')]
+        nodes_sorted = sorted(list(nodes), key=sort_key_func)
+        nodes_mapper = {n.key: n for n in nodes_sorted}
+        if not self.is_org_root():
+            # 如果是org_root，那么parent_key为'', parent为自己，所以这种情况不处理
+            # 更新自己时，自己的parent_key获取不到
+            nodes_mapper.update({self.parent_key: self.parent})
+        for node in nodes_sorted:
+            parent = nodes_mapper.get(node.parent_key)
+            if not parent:
+                if node.parent_key:
+                    logger.error(f'Node parent node in mapper: {node.parent_key} {node.value}')
+                continue
+            node.full_value = parent.full_value + '/' + node.value
+        self.__class__.objects.bulk_update(nodes, ['full_value'])
+
+    def save(self, *args, **kwargs):
+        self.full_value = self.computed_full_value()
+        instance = super().save(*args, **kwargs)
+        self.update_child_full_value()
+        return instance

apps/assets/models/user.py

@@ -72,6 +72,9 @@ class SystemUser(BaseUser):
     PROTOCOL_TELNET = 'telnet'
     PROTOCOL_VNC = 'vnc'
     PROTOCOL_MYSQL = 'mysql'
+    PROTOCOL_ORACLE = 'oracle'
+    PROTOCOL_MARIADB = 'mariadb'
+    PROTOCOL_POSTGRESQL = 'postgresql'
     PROTOCOL_K8S = 'k8s'
     PROTOCOL_CHOICES = (
         (PROTOCOL_SSH, 'ssh'),
@@ -79,8 +82,28 @@ class SystemUser(BaseUser):
         (PROTOCOL_TELNET, 'telnet'),
         (PROTOCOL_VNC, 'vnc'),
         (PROTOCOL_MYSQL, 'mysql'),
+        (PROTOCOL_ORACLE, 'oracle'),
+        (PROTOCOL_MARIADB, 'mariadb'),
+        (PROTOCOL_POSTGRESQL, 'postgresql'),
         (PROTOCOL_K8S, 'k8s'),
     )
+    ASSET_CATEGORY_PROTOCOLS = [
+        PROTOCOL_SSH, PROTOCOL_RDP, PROTOCOL_TELNET, PROTOCOL_VNC
+    ]
+    APPLICATION_CATEGORY_REMOTE_APP_PROTOCOLS = [
+        PROTOCOL_RDP
+    ]
+    APPLICATION_CATEGORY_DB_PROTOCOLS = [
+        PROTOCOL_MYSQL, PROTOCOL_ORACLE, PROTOCOL_MARIADB, PROTOCOL_POSTGRESQL
+    ]
+    APPLICATION_CATEGORY_CLOUD_PROTOCOLS = [
+        PROTOCOL_K8S
+    ]
+    APPLICATION_CATEGORY_PROTOCOLS = [
+        *APPLICATION_CATEGORY_REMOTE_APP_PROTOCOLS,
+        *APPLICATION_CATEGORY_DB_PROTOCOLS,
+        *APPLICATION_CATEGORY_CLOUD_PROTOCOLS
+    ]
 
     LOGIN_AUTO = 'auto'
     LOGIN_MANUAL = 'manual'
@@ -104,6 +127,7 @@ class SystemUser(BaseUser):
     token = models.TextField(default='', verbose_name=_('Token'))
     home = models.CharField(max_length=4096, default='', verbose_name=_('Home'), blank=True)
     system_groups = models.CharField(default='', max_length=4096, verbose_name=_('System groups'), blank=True)
+    ad_domain = models.CharField(default='', max_length=256)
     _prefer = 'system_user'
 
     def __str__(self):
@@ -138,11 +162,16 @@ class SystemUser(BaseUser):
 
     @property
     def is_need_test_asset_connective(self):
-        return self.protocol not in [self.PROTOCOL_MYSQL]
+        return self.protocol in self.ASSET_CATEGORY_PROTOCOLS
+
+    def has_special_auth(self, asset=None, username=None):
+        if username is None and self.username_same_with_user:
+            raise TypeError('System user is dynamic, username should be pass')
+        return super().has_special_auth(asset=asset, username=username)
 
     @property
     def can_perm_to_asset(self):
-        return self.protocol not in [self.PROTOCOL_MYSQL]
+        return self.protocol in self.ASSET_CATEGORY_PROTOCOLS
 
     def _merge_auth(self, other):
         super()._merge_auth(other)
@@ -175,6 +204,17 @@ class SystemUser(BaseUser):
         assets = Asset.objects.filter(id__in=assets_ids)
         return assets
 
+    @classmethod
+    def get_protocol_by_application_type(cls, app_type):
+        from applications.const import ApplicationTypeChoices
+        if app_type in cls.APPLICATION_CATEGORY_PROTOCOLS:
+            protocol = app_type
+        elif app_type in ApplicationTypeChoices.remote_app_types():
+            protocol = cls.PROTOCOL_RDP
+        else:
+            protocol = None
+        return protocol
+
     class Meta:
         ordering = ['name']
         unique_together = [('name', 'org_id')]

apps/assets/serializers/admin_user.py

@@ -3,7 +3,7 @@
 from django.utils.translation import ugettext_lazy as _
 from rest_framework import serializers
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 
 from ..models import Node, AdminUser
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer

apps/assets/serializers/asset.py

@@ -1,13 +1,12 @@
 # -*- coding: utf-8 -*-
 #
 from rest_framework import serializers
-from django.db.models import Prefetch, F, Count
-
+from django.db.models import F
+from django.core.validators import RegexValidator
 from django.utils.translation import ugettext_lazy as _
 
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-from common.serializers import AdaptedBulkListSerializer
-from ..models import Asset, Node, Label, Platform
+from ..models import Asset, Node, Platform
 from .base import ConnectivitySerializer
 
 __all__ = [
@@ -67,8 +66,9 @@ class AssetSerializer(BulkOrgResourceModelSerializer):
         slug_field='name', queryset=Platform.objects.all(), label=_("Platform")
     )
     protocols = ProtocolsField(label=_('Protocols'), required=False)
-    domain_display = serializers.ReadOnlyField(source='domain.name')
-    admin_user_display = serializers.ReadOnlyField(source='admin_user.name')
+    domain_display = serializers.ReadOnlyField(source='domain.name', label=_('Domain name'))
+    admin_user_display = serializers.ReadOnlyField(source='admin_user.name', label=_('Admin user name'))
+    nodes_display = serializers.ListField(child=serializers.CharField(), label=_('Nodes name'), required=False)
 
     """
     资产的数据结构
@@ -90,7 +90,7 @@ class AssetSerializer(BulkOrgResourceModelSerializer):
             'platform': ['name']
         }
         fields_m2m = [
-            'nodes', 'labels',
+            'nodes', 'nodes_display', 'labels',
         ]
         annotates_fields = {
             # 'admin_user_display': 'admin_user__name'
@@ -98,9 +98,6 @@ class AssetSerializer(BulkOrgResourceModelSerializer):
         fields_as = list(annotates_fields.keys())
         fields = fields_small + fields_fk + fields_m2m + fields_as
         read_only_fields = [
-            'vendor', 'model', 'sn', 'cpu_model', 'cpu_count',
-            'cpu_cores', 'cpu_vcpus', 'memory', 'disk_total', 'disk_info',
-            'os', 'os_version', 'os_arch', 'hostname_raw',
             'created_by', 'date_created',
         ] + fields_as
 
@@ -133,14 +130,32 @@ class AssetSerializer(BulkOrgResourceModelSerializer):
         if protocols_data:
             validated_data["protocols"] = ' '.join(protocols_data)
 
+    def perform_nodes_display_create(self, instance, nodes_display):
+        if not nodes_display:
+            return
+        nodes_to_set = []
+        for full_value in nodes_display:
+            node = Node.objects.filter(full_value=full_value).first()
+            if node:
+                nodes_to_set.append(node)
+            else:
+                node = Node.create_node_by_full_value(full_value)
+            nodes_to_set.append(node)
+        instance.nodes.set(nodes_to_set)
+
     def create(self, validated_data):
         self.compatible_with_old_protocol(validated_data)
+        nodes_display = validated_data.pop('nodes_display', '')
         instance = super().create(validated_data)
+        self.perform_nodes_display_create(instance, nodes_display)
         return instance
 
     def update(self, instance, validated_data):
+        nodes_display = validated_data.pop('nodes_display', '')
         self.compatible_with_old_protocol(validated_data)
-        return super().update(instance, validated_data)
+        instance = super().update(instance, validated_data)
+        self.perform_nodes_display_create(instance, nodes_display)
+        return instance
 
 
 class AssetDisplaySerializer(AssetSerializer):
@@ -162,6 +177,14 @@ class AssetDisplaySerializer(AssetSerializer):
 class PlatformSerializer(serializers.ModelSerializer):
     meta = serializers.DictField(required=False, allow_null=True)
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+        # TODO 修复 drf SlugField RegexValidator bug，之后记得删除
+        validators = self.fields['name'].validators
+        if isinstance(validators[-1], RegexValidator):
+            validators.pop()
+
     class Meta:
         model = Platform
         fields = [
@@ -189,3 +212,6 @@ class AssetTaskSerializer(serializers.Serializer):
     )
     task = serializers.CharField(read_only=True)
     action = serializers.ChoiceField(choices=ACTION_CHOICES, write_only=True)
+    assets = serializers.PrimaryKeyRelatedField(
+        queryset=Asset.objects, required=False, allow_empty=True, many=True
+    )

apps/assets/serializers/asset_user.py

@@ -4,7 +4,7 @@
 from django.utils.translation import ugettext as _
 from rest_framework import serializers
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 from ..models import AuthBook, Asset
 from ..backends import AssetUserManager

apps/assets/serializers/cmd_filter.py

@@ -3,8 +3,7 @@
 import re
 from rest_framework import serializers
 
-from common.fields import ChoiceDisplayField
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from ..models import CommandFilter, CommandFilterRule, SystemUser
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 
@@ -26,7 +25,6 @@ class CommandFilterSerializer(BulkOrgResourceModelSerializer):
 
 
 class CommandFilterRuleSerializer(BulkOrgResourceModelSerializer):
-    # serializer_choice_field = ChoiceDisplayField
     invalid_pattern = re.compile(r'[\.\*\+\[\\\?\{\}\^\$\|\(\)\#\<\>]')
     type_display = serializers.ReadOnlyField(source='get_type_display')
     action_display = serializers.ReadOnlyField(source='get_action_display')

apps/assets/serializers/domain.py

@@ -1,17 +1,19 @@
 # -*- coding: utf-8 -*-
 #
 from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-
+from common.validators import NoSpecialChars
 from ..models import Domain, Gateway
 from .base import AuthSerializerMixin
 
 
 class DomainSerializer(BulkOrgResourceModelSerializer):
-    asset_count = serializers.SerializerMethodField()
-    gateway_count = serializers.SerializerMethodField()
+    asset_count = serializers.SerializerMethodField(label=_('Assets count'))
+    application_count = serializers.SerializerMethodField(label=_('Applications count'))
+    gateway_count = serializers.SerializerMethodField(label=_('Gateways count'))
 
     class Meta:
         model = Domain
@@ -20,12 +22,12 @@ class DomainSerializer(BulkOrgResourceModelSerializer):
             'comment', 'date_created'
         ]
         fields_m2m = [
-            'asset_count', 'assets', 'gateway_count',
+            'asset_count', 'assets', 'application_count', 'gateway_count',
         ]
         fields = fields_small + fields_m2m
         read_only_fields = ('asset_count', 'gateway_count', 'date_created')
         extra_kwargs = {
-            'assets': {'required': False}
+            'assets': {'required': False, 'label': _('Assets')},
         }
         list_serializer_class = AdaptedBulkListSerializer
 
@@ -33,6 +35,10 @@ class DomainSerializer(BulkOrgResourceModelSerializer):
     def get_asset_count(obj):
         return obj.assets.count()
 
+    @staticmethod
+    def get_application_count(obj):
+        return obj.applications.count()
+
     @staticmethod
     def get_gateway_count(obj):
         return obj.gateway_set.all().count()
@@ -47,6 +53,9 @@ class GatewaySerializer(AuthSerializerMixin, BulkOrgResourceModelSerializer):
             'private_key', 'public_key', 'domain', 'is_active', 'date_created',
             'date_updated', 'created_by', 'comment',
         ]
+        extra_kwargs = {
+            'password': {'validators': [NoSpecialChars()]}
+        }
 
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)

apps/assets/serializers/favorite_asset.py

@@ -4,7 +4,7 @@
 from rest_framework import serializers
 
 from orgs.utils import tmp_to_root_org
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from common.mixins import BulkSerializerMixin
 from ..models import FavoriteAsset
 

apps/assets/serializers/label.py

@@ -2,7 +2,7 @@
 #
 from rest_framework import serializers
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 
 from ..models import Label

apps/assets/serializers/node.py

@@ -25,6 +25,9 @@ class NodeSerializer(BulkOrgResourceModelSerializer):
         read_only_fields = ['key', 'org_id']
 
     def validate_value(self, data):
+        if '/' in data:
+            error = _("Can't contains: " + "/")
+            raise serializers.ValidationError(error)
         if self.instance:
             instance = self.instance
             siblings = instance.get_siblings()

apps/assets/serializers/system_user.py

@@ -2,11 +2,10 @@ from rest_framework import serializers
 from django.utils.translation import ugettext_lazy as _
 from django.db.models import Count
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from common.mixins.serializers import BulkSerializerMixin
 from common.utils import ssh_pubkey_gen
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-from assets.models import Node
 from ..models import SystemUser, Asset
 from .base import AuthSerializerMixin
 
@@ -35,17 +34,18 @@ class SystemUserSerializer(AuthSerializerMixin, BulkOrgResourceModelSerializer):
             'auto_push', 'cmd_filters', 'sudo', 'shell', 'comment',
             'auto_generate_key', 'sftp_root', 'token',
             'assets_amount', 'date_created', 'created_by',
-            'home', 'system_groups'
+            'home', 'system_groups', 'ad_domain'
         ]
         extra_kwargs = {
             'password': {"write_only": True},
             'public_key': {"write_only": True},
             'private_key': {"write_only": True},
             'token': {"write_only": True},
-            'nodes_amount': {'label': _('Node')},
-            'assets_amount': {'label': _('Asset')},
+            'nodes_amount': {'label': _('Nodes amount')},
+            'assets_amount': {'label': _('Assets amount')},
             'login_mode_display': {'label': _('Login mode display')},
             'created_by': {'read_only': True},
+            'ad_domain': {'required': False, 'allow_blank': True, 'label': _('Ad domain')},
         }
 
     def validate_auto_push(self, value):
@@ -154,14 +154,18 @@ class SystemUserListSerializer(SystemUserSerializer):
             'priority', "username_same_with_user",
             'auto_push', 'sudo', 'shell', 'comment',
             "assets_amount", 'home', 'system_groups',
-            'auto_generate_key',
+            'auto_generate_key', 'ad_domain',
             'sftp_root',
         ]
-
         extra_kwargs = {
             'password': {"write_only": True},
             'public_key': {"write_only": True},
             'private_key': {"write_only": True},
+            'nodes_amount': {'label': _('Nodes amount')},
+            'assets_amount': {'label': _('Assets amount')},
+            'login_mode_display': {'label': _('Login mode display')},
+            'created_by': {'read_only': True},
+            'ad_domain': {'label': _('Ad domain')},
         }
 
     @classmethod
@@ -179,7 +183,8 @@ class SystemUserWithAuthInfoSerializer(SystemUserSerializer):
             'login_mode', 'login_mode_display',
             'priority', 'username_same_with_user',
             'auto_push', 'sudo', 'shell', 'comment',
-            'auto_generate_key', 'sftp_root', 'token'
+            'auto_generate_key', 'sftp_root', 'token',
+            'ad_domain',
         ]
         extra_kwargs = {
             'nodes_amount': {'label': _('Node')},
@@ -252,4 +257,8 @@ class SystemUserTaskSerializer(serializers.Serializer):
     asset = serializers.PrimaryKeyRelatedField(
         queryset=Asset.objects, allow_null=True, required=False, write_only=True
     )
+    assets = serializers.PrimaryKeyRelatedField(
+        queryset=Asset.objects, allow_null=True, required=False, write_only=True,
+        many=True
+    )
     task = serializers.CharField(read_only=True)

apps/assets/signals_handler.py

@@ -4,7 +4,7 @@ from operator import add, sub
 
 from assets.utils import is_asset_exists_in_node
 from django.db.models.signals import (
-    post_save, m2m_changed, pre_delete, post_delete
+    post_save, m2m_changed, pre_delete, post_delete, pre_save
 )
 from django.db.models import Q, F
 from django.dispatch import receiver
@@ -37,6 +37,11 @@ def test_asset_conn_on_created(asset):
     test_asset_connectivity_util.delay([asset])
 
 
+@receiver(pre_save, sender=Node)
+def on_node_pre_save(sender, instance: Node, **kwargs):
+    instance.parent_key = instance.compute_parent_key()
+
+
 @receiver(post_save, sender=Asset)
 @on_transaction_commit
 def on_asset_created_or_update(sender, instance=None, created=False, **kwargs):
@@ -58,7 +63,8 @@ def on_asset_created_or_update(sender, instance=None, created=False, **kwargs):
 
 
 @receiver(post_save, sender=SystemUser, dispatch_uid="jms")
-def on_system_user_update(sender, instance=None, created=True, **kwargs):
+@on_transaction_commit
+def on_system_user_update(instance: SystemUser, created, **kwargs):
     """
     当系统用户更新时，可能更新了秘钥，用户名等，这时要自动推送系统用户到资产上,
     其实应该当 用户名，密码，秘钥 sudo等更新时再推送，这里偷个懒,
@@ -68,48 +74,52 @@ def on_system_user_update(sender, instance=None, created=True, **kwargs):
     if instance and not created:
         logger.info("System user update signal recv: {}".format(instance))
         assets = instance.assets.all().valid()
-        push_system_user_to_assets.delay(instance, assets)
+        push_system_user_to_assets.delay(instance.id, [_asset.id for _asset in assets])
 
 
 @receiver(m2m_changed, sender=SystemUser.assets.through)
-def on_system_user_assets_change(sender, instance=None, action='', model=None, pk_set=None, **kwargs):
+@on_transaction_commit
+def on_system_user_assets_change(instance, action, model, pk_set, **kwargs):
     """
     当系统用户和资产关系发生变化时，应该重新推送系统用户到新添加的资产中
     """
     if action != POST_ADD:
         return
     logger.debug("System user assets change signal recv: {}".format(instance))
-    queryset = model.objects.filter(pk__in=pk_set)
     if model == Asset:
-        system_users = [instance]
-        assets = queryset
+        system_users_id = [instance.id]
+        assets_id = pk_set
     else:
-        system_users = queryset
-        assets = [instance]
-    for system_user in system_users:
-        push_system_user_to_assets.delay(system_user, assets)
+        system_users_id = pk_set
+        assets_id = [instance.id]
+    for system_user_id in system_users_id:
+        push_system_user_to_assets.delay(system_user_id, assets_id)
 
 
 @receiver(m2m_changed, sender=SystemUser.users.through)
-def on_system_user_users_change(sender, instance=None, action='', model=None, pk_set=None, **kwargs):
+@on_transaction_commit
+def on_system_user_users_change(sender, instance: SystemUser, action, model, pk_set, reverse, **kwargs):
     """
     当系统用户和用户关系发生变化时，应该重新推送系统用户资产中
     """
     if action != POST_ADD:
         return
+
+    if reverse:
+        raise M2MReverseNotAllowed
+
     if not instance.username_same_with_user:
         return
+
     logger.debug("System user users change signal recv: {}".format(instance))
-    queryset = model.objects.filter(pk__in=pk_set)
-    if model == SystemUser:
-        system_users = queryset
-    else:
-        system_users = [instance]
-    for s in system_users:
-        push_system_user_to_assets_manual.delay(s)
+    usernames = model.objects.filter(pk__in=pk_set).values_list('username', flat=True)
+
+    for username in usernames:
+        push_system_user_to_assets_manual.delay(instance, username)
 
 
 @receiver(m2m_changed, sender=SystemUser.nodes.through)
+@on_transaction_commit
 def on_system_user_nodes_change(sender, instance=None, action=None, model=None, pk_set=None, **kwargs):
     """
     当系统用户和节点关系发生变化时，应该将节点下资产关联到新的系统用户上
@@ -140,7 +150,7 @@ def on_system_user_groups_change(instance, action, pk_set, reverse, **kwargs):
     logger.info("System user groups update signal recv: {}".format(instance))
 
     users = User.objects.filter(groups__id__in=pk_set).distinct()
-    instance.users.add(users)
+    instance.users.add(*users)
 
 
 @receiver(m2m_changed, sender=Asset.nodes.through)

apps/assets/tasks/asset_connectivity.py

@@ -14,7 +14,7 @@ from .utils import clean_ansible_task_hosts, group_asset_by_platform
 logger = get_logger(__file__)
 __all__ = [
     'test_asset_connectivity_util', 'test_asset_connectivity_manual',
-    'test_node_assets_connectivity_manual',
+    'test_node_assets_connectivity_manual', 'test_assets_connectivity_manual',
 ]
 
 
@@ -82,6 +82,17 @@ def test_asset_connectivity_manual(asset):
         return True, ""
 
 
+@shared_task(queue="ansible")
+def test_assets_connectivity_manual(assets):
+    task_name = _("Test assets connectivity: {}").format([asset.hostname for asset in assets])
+    summary = test_asset_connectivity_util(assets, task_name=task_name)
+
+    if summary.get('dark'):
+        return False, summary['dark']
+    else:
+        return True, ""
+
+
 @shared_task(queue="ansible")
 def test_node_assets_connectivity_manual(node):
     task_name = _("Test if the assets under the node are connectable: {}".format(node.name))

apps/assets/tasks/common.py

@@ -3,10 +3,13 @@
 
 from celery import shared_task
 
+from orgs.utils import tmp_to_root_org
+
 __all__ = ['add_nodes_assets_to_system_users']
 
 
 @shared_task
+@tmp_to_root_org()
 def add_nodes_assets_to_system_users(nodes_keys, system_users):
     from ..models import Node
     assets = Node.get_nodes_all_assets(nodes_keys).values_list('id', flat=True)

apps/assets/tasks/gather_asset_hardware_info.py

@@ -19,6 +19,7 @@ disk_pattern = re.compile(r'^hd|sd|xvd|vd|nv')
 __all__ = [
     'update_assets_hardware_info_util', 'update_asset_hardware_info_manual',
     'update_assets_hardware_info_period',  'update_node_assets_hardware_info_manual',
+    'update_assets_hardware_info_manual',
 ]
 
 
@@ -114,6 +115,12 @@ def update_asset_hardware_info_manual(asset):
     update_assets_hardware_info_util([asset], task_name=task_name)
 
 
+@shared_task(queue="ansible")
+def update_assets_hardware_info_manual(assets):
+    task_name = _("Update assets hardware info: {}").format([asset.hostname for asset in assets])
+    update_assets_hardware_info_util(assets, task_name=task_name)
+
+
 @shared_task(queue="ansible")
 def update_assets_hardware_info_period():
     """

apps/assets/tasks/nodes_amount.py

@@ -1,14 +1,27 @@
 from celery import shared_task
+from django.utils.translation import gettext_lazy as _
 
+from orgs.models import Organization
+from orgs.utils import tmp_to_org
+from ops.celery.decorator import register_as_period_task
 from assets.utils import check_node_assets_amount
+
+from common.utils.lock import AcquireFailed
 from common.utils import get_logger
-from common.utils.timezone import now
 
 logger = get_logger(__file__)
 
 
-@shared_task()
-def check_node_assets_amount_celery_task():
-    logger.info(f'>>> {now()} begin check_node_assets_amount_celery_task ...')
-    check_node_assets_amount()
-    logger.info(f'>>> {now()} end check_node_assets_amount_celery_task ...')
+@shared_task(queue='celery_heavy_tasks')
+def check_node_assets_amount_task(org_id=Organization.ROOT_ID):
+    try:
+        with tmp_to_org(Organization.get_instance(org_id)):
+            check_node_assets_amount()
+    except AcquireFailed:
+        logger.error(_('The task of self-checking is already running and cannot be started repeatedly'))
+
+
+@register_as_period_task(crontab='0 2 * * *')
+@shared_task(queue='celery_heavy_tasks')
+def check_node_assets_amount_period_task():
+    check_node_assets_amount_task()

apps/assets/tasks/push_system_user.py

@@ -2,13 +2,13 @@
 
 from itertools import groupby
 from celery import shared_task
-from common.db.utils import get_object_if_need, get_objects_if_need
+from common.db.utils import get_object_if_need, get_objects
 from django.utils.translation import ugettext as _
 from django.db.models import Empty
 
 from common.utils import encrypt_password, get_logger
-from assets.models import SystemUser, Asset
-from orgs.utils import org_aware_func
+from assets.models import SystemUser, Asset, AuthBook
+from orgs.utils import org_aware_func, tmp_to_root_org
 from . import const
 from .utils import clean_ansible_task_hosts, group_asset_by_platform
 
@@ -36,6 +36,7 @@ def get_push_unixlike_system_user_tasks(system_user, username=None):
         username = system_user.username
     password = system_user.password
     public_key = system_user.public_key
+    comment = system_user.name
 
     groups = _split_by_comma(system_user.system_groups)
 
@@ -47,7 +48,8 @@ def get_push_unixlike_system_user_tasks(system_user, username=None):
         'shell': system_user.shell or Empty,
         'state': 'present',
         'home': system_user.home or Empty,
-        'groups': groups or Empty
+        'groups': groups or Empty,
+        'comment': comment
     }
 
     tasks = [
@@ -64,24 +66,27 @@ def get_push_unixlike_system_user_tasks(system_user, username=None):
                 'module': 'group',
                 'args': 'name={} state=present'.format(username),
             }
-        },
-        {
-            'name': 'Check home dir exists',
-            'action': {
-                'module': 'stat',
-                'args': 'path=/home/{}'.format(username)
-            },
-            'register': 'home_existed'
-        },
-        {
-            'name': "Set home dir permission",
-            'action': {
-                'module': 'file',
-                'args': "path=/home/{0} owner={0} group={0} mode=700".format(username)
-            },
-            'when': 'home_existed.stat.exists == true'
         }
     ]
+    if not system_user.home:
+        tasks.extend([
+            {
+                'name': 'Check home dir exists',
+                'action': {
+                    'module': 'stat',
+                    'args': 'path=/home/{}'.format(username)
+                },
+                'register': 'home_existed'
+            },
+            {
+                'name': "Set home dir permission",
+                'action': {
+                    'module': 'file',
+                    'args': "path=/home/{0} owner={0} group={0} mode=700".format(username)
+                },
+                'when': 'home_existed.stat.exists == true'
+            }
+        ])
     if password:
         tasks.append({
             'name': 'Set {} password'.format(username),
@@ -134,6 +139,7 @@ def get_push_windows_system_user_tasks(system_user, username=None):
 
     tasks = []
     if not password:
+        logger.error("Error: no password found")
         return tasks
     task = {
         'name': 'Add user {}'.format(username),
@@ -184,15 +190,12 @@ def get_push_system_user_tasks(system_user, platform="unixlike", username=None):
 @org_aware_func("system_user")
 def push_system_user_util(system_user, assets, task_name, username=None):
     from ops.utils import update_or_create_ansible_task
-    hosts = clean_ansible_task_hosts(assets, system_user=system_user)
-    if not hosts:
+    assets = clean_ansible_task_hosts(assets, system_user=system_user)
+    if not assets:
         return {}
 
-    platform_hosts_map = {}
-    hosts_sorted = sorted(hosts, key=group_asset_by_platform)
-    platform_hosts = groupby(hosts_sorted, key=group_asset_by_platform)
-    for i in platform_hosts:
-        platform_hosts_map[i[0]] = list(i[1])
+    assets_sorted = sorted(assets, key=group_asset_by_platform)
+    platform_hosts = groupby(assets_sorted, key=group_asset_by_platform)
 
     def run_task(_tasks, _hosts):
         if not _tasks:
@@ -203,26 +206,59 @@ def push_system_user_util(system_user, assets, task_name, username=None):
         )
         task.run()
 
-    for platform, _hosts in platform_hosts_map.items():
-        if not _hosts:
+    if system_user.username_same_with_user:
+        if username is None:
+            # 动态系统用户，但是没有指定 username
+            usernames = list(system_user.users.all().values_list('username', flat=True).distinct())
+        else:
+            usernames = [username]
+    else:
+        # 非动态系统用户指定 username 无效
+        assert username is None, 'Only Dynamic user can assign `username`'
+        usernames = [system_user.username]
+
+    for platform, _assets in platform_hosts:
+        _assets = list(_assets)
+        if not _assets:
             continue
         print(_("Start push system user for platform: [{}]").format(platform))
-        print(_("Hosts count: {}").format(len(_hosts)))
+        print(_("Hosts count: {}").format(len(_assets)))
 
-        if not system_user.has_special_auth():
-            logger.debug("System user not has special auth")
-            tasks = get_push_system_user_tasks(system_user, platform, username=username)
-            run_task(tasks, _hosts)
-            continue
+        id_asset_map = {_asset.id: _asset for _asset in _assets}
+        assets_id = id_asset_map.keys()
+        no_special_auth = []
+        special_auth_set = set()
 
-        for _host in _hosts:
-            system_user.load_asset_special_auth(_host)
-            tasks = get_push_system_user_tasks(system_user, platform, username=username)
-            run_task(tasks, [_host])
+        auth_books = AuthBook.objects.filter(username__in=usernames, asset_id__in=assets_id)
+
+        for auth_book in auth_books:
+            special_auth_set.add((auth_book.username, auth_book.asset_id))
+
+        for _username in usernames:
+            no_special_assets = []
+            for asset_id in assets_id:
+                if (_username, asset_id) not in special_auth_set:
+                    no_special_assets.append(id_asset_map[asset_id])
+            if no_special_assets:
+                no_special_auth.append((_username, no_special_assets))
+
+        for _username, no_special_assets in no_special_auth:
+            tasks = get_push_system_user_tasks(system_user, platform, username=_username)
+            run_task(tasks, no_special_assets)
+
+        for auth_book in auth_books:
+            system_user._merge_auth(auth_book)
+            tasks = get_push_system_user_tasks(system_user, platform, username=auth_book.username)
+            asset = id_asset_map[auth_book.asset_id]
+            run_task(tasks, [asset])
 
 
 @shared_task(queue="ansible")
+@tmp_to_root_org()
 def push_system_user_to_assets_manual(system_user, username=None):
+    """
+    将系统用户推送到与它关联的所有资产上
+    """
     system_user = get_object_if_need(SystemUser, system_user)
     assets = system_user.get_related_assets()
     task_name = _("Push system users to assets: {}").format(system_user.name)
@@ -230,7 +266,11 @@ def push_system_user_to_assets_manual(system_user, username=None):
 
 
 @shared_task(queue="ansible")
+@tmp_to_root_org()
 def push_system_user_a_asset_manual(system_user, asset, username=None):
+    """
+    将系统用户推送到一个资产上
+    """
     if username is None:
         username = system_user.username
     task_name = _("Push system users to asset: {}({}) => {}").format(
@@ -240,10 +280,15 @@ def push_system_user_a_asset_manual(system_user, asset, username=None):
 
 
 @shared_task(queue="ansible")
-def push_system_user_to_assets(system_user, assets, username=None):
+@tmp_to_root_org()
+def push_system_user_to_assets(system_user_id, assets_id, username=None):
+    """
+    推送系统用户到指定的若干资产上
+    """
+    system_user = SystemUser.objects.get(id=system_user_id)
+    assets = get_objects(Asset, assets_id)
     task_name = _("Push system users to assets: {}").format(system_user.name)
-    system_user = get_object_if_need(SystemUser, system_user)
-    assets = get_objects_if_need(Asset, assets)
+
     return push_system_user_util(system_user, assets, task_name, username=username)
 
 # @shared_task

apps/assets/urls/api_urls.py

@@ -36,6 +36,7 @@ urlpatterns = [
     path('assets/<uuid:pk>/gateways/', api.AssetGatewayListApi.as_view(), name='asset-gateway-list'),
     path('assets/<uuid:pk>/platform/', api.AssetPlatformRetrieveApi.as_view(), name='asset-platform-detail'),
     path('assets/<uuid:pk>/tasks/', api.AssetTaskCreateApi.as_view(), name='asset-task-create'),
+    path('assets/tasks/', api.AssetsTaskCreateApi.as_view(), name='assets-task-create'),
 
     path('asset-users/tasks/', api.AssetUserTaskCreateAPI.as_view(), name='asset-user-task-create'),
 
@@ -45,6 +46,7 @@ urlpatterns = [
     path('admin-users/<uuid:pk>/assets/', api.AdminUserAssetsListView.as_view(), name='admin-user-assets'),
 
     path('system-users/<uuid:pk>/auth-info/', api.SystemUserAuthInfoApi.as_view(), name='system-user-auth-info'),
+    path('system-users/<uuid:pk>/assets/', api.SystemUserAssetsListView.as_view(), name='system-user-assets'),
     path('system-users/<uuid:pk>/assets/<uuid:aid>/auth-info/', api.SystemUserAssetAuthInfoApi.as_view(), name='system-user-asset-auth-info'),
     path('system-users/<uuid:pk>/tasks/', api.SystemUserTaskApi.as_view(), name='system-user-task-create'),
     path('system-users/<uuid:pk>/cmd-filter-rules/', api.SystemUserCommandFilterRuleListApi.as_view(), name='system-user-cmd-filter-rule-list'),

apps/assets/utils.py

@@ -1,8 +1,11 @@
 # ~*~ coding: utf-8 ~*~
 #
+import time
+
 from django.db.models import Q
 
 from common.utils import get_logger, dict_get_any, is_uuid, get_object_or_none
+from common.utils.lock import DistributedLock
 from common.http import is_true
 from .models import Asset, Node
 
@@ -10,17 +13,21 @@ from .models import Asset, Node
 logger = get_logger(__file__)
 
 
+@DistributedLock(name="assets.node.check_node_assets_amount", blocking=False)
 def check_node_assets_amount():
     for node in Node.objects.all():
+        logger.info(f'Check node assets amount: {node}')
         assets_amount = Asset.objects.filter(
             Q(nodes__key__istartswith=f'{node.key}:') | Q(nodes=node)
         ).distinct().count()
 
         if node.assets_amount != assets_amount:
-            print(f'>>> <Node:{node.key}> wrong assets amount '
-                  f'{node.assets_amount} right is {assets_amount}')
+            logger.warn(f'Node wrong assets amount <Node:{node.key}> '
+                        f'{node.assets_amount} right is {assets_amount}')
             node.assets_amount = assets_amount
             node.save()
+        # 防止自检程序给数据库的压力太大
+        time.sleep(0.1)
 
 
 def is_asset_exists_in_node(asset_pk, node_key):
@@ -33,7 +40,7 @@ def is_asset_exists_in_node(asset_pk, node_key):
 
 def is_query_node_all_assets(request):
     request = request
-    query_all_arg = request.query_params.get('all')
+    query_all_arg = request.query_params.get('all', 'true')
     show_current_asset_arg = request.query_params.get('show_current_asset')
     if show_current_asset_arg is not None:
         return not is_true(show_current_asset_arg)

apps/audits/api.py

@@ -25,8 +25,8 @@ class FTPLogViewSet(CreateModelMixin,
     date_range_filter_fields = [
         ('date_start', ('date_from', 'date_to'))
     ]
-    filter_fields = ['user', 'asset', 'system_user', 'filename']
-    search_fields = filter_fields
+    filterset_fields = ['user', 'asset', 'system_user', 'filename']
+    search_fields = filterset_fields
     ordering = ['-date_start']
 
 
@@ -38,7 +38,7 @@ class UserLoginLogViewSet(ListModelMixin, CommonGenericViewSet):
     date_range_filter_fields = [
         ('datetime', ('date_from', 'date_to'))
     ]
-    filter_fields = ['username', 'ip', 'city', 'type', 'status', 'mfa']
+    filterset_fields = ['username', 'ip', 'city', 'type', 'status', 'mfa']
     search_fields =['username', 'ip', 'city']
 
     @staticmethod
@@ -62,7 +62,7 @@ class OperateLogViewSet(ListModelMixin, OrgGenericViewSet):
     date_range_filter_fields = [
         ('datetime', ('date_from', 'date_to'))
     ]
-    filter_fields = ['user', 'action', 'resource_type', 'resource', 'remote_addr']
+    filterset_fields = ['user', 'action', 'resource_type', 'resource', 'remote_addr']
     search_fields = ['resource']
     ordering = ['-datetime']
 
@@ -75,7 +75,7 @@ class PasswordChangeLogViewSet(ListModelMixin, CommonGenericViewSet):
     date_range_filter_fields = [
         ('datetime', ('date_from', 'date_to'))
     ]
-    filter_fields = ['user', 'change_by', 'remote_addr']
+    filterset_fields = ['user', 'change_by', 'remote_addr']
     ordering = ['-datetime']
 
     def get_queryset(self):
@@ -94,7 +94,7 @@ class CommandExecutionViewSet(ListModelMixin, OrgGenericViewSet):
     date_range_filter_fields = [
         ('date_start', ('date_from', 'date_to'))
     ]
-    filter_fields = ['user__name', 'command', 'run_as__name', 'is_finished']
+    filterset_fields = ['user__name', 'command', 'run_as__name', 'is_finished']
     search_fields = ['command', 'user__name', 'run_as__name']
     ordering = ['-date_created']
 
@@ -108,7 +108,7 @@ class CommandExecutionHostRelationViewSet(OrgRelationMixin, OrgBulkModelViewSet)
     serializer_class = CommandExecutionHostsRelationSerializer
     m2m_field = CommandExecution.hosts.field
     permission_classes = [IsOrgAdmin | IsOrgAuditor]
-    filter_fields = [
+    filterset_fields = [
         'id', 'asset', 'commandexecution'
     ]
     search_fields = ('asset__hostname', )

apps/audits/migrations/0011_userloginlog_backend.py

@@ -0,0 +1,18 @@
+# Generated by Django 3.1 on 2020-12-09 03:03
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('audits', '0010_auto_20200811_1122'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='userloginlog',
+            name='backend',
+            field=models.CharField(default='', max_length=32, verbose_name='Authentication backend'),
+        ),
+    ]

apps/audits/models.py

@@ -105,6 +105,7 @@ class UserLoginLog(models.Model):
     reason = models.CharField(default='', max_length=128, blank=True, verbose_name=_('Reason'))
     status = models.BooleanField(max_length=2, default=True, choices=STATUS_CHOICE, verbose_name=_('Status'))
     datetime = models.DateTimeField(default=timezone.now, verbose_name=_('Date login'))
+    backend = models.CharField(max_length=32, default='', verbose_name=_('Authentication backend'))
 
     @classmethod
     def get_login_logs(cls, date_from=None, date_to=None, user=None, keyword=None):

apps/audits/serializers.py

@@ -5,14 +5,14 @@ from rest_framework import serializers
 from django.db.models import F
 
 from common.mixins import BulkSerializerMixin
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from terminal.models import Session
 from ops.models import CommandExecution
 from . import models
 
 
 class FTPLogSerializer(serializers.ModelSerializer):
-    operate_display = serializers.ReadOnlyField(source='get_operate_display')
+    operate_display = serializers.ReadOnlyField(source='get_operate_display', label=_('Operate for display'))
 
     class Meta:
         model = models.FTPLog
@@ -23,16 +23,20 @@ class FTPLogSerializer(serializers.ModelSerializer):
 
 
 class UserLoginLogSerializer(serializers.ModelSerializer):
-    type_display = serializers.ReadOnlyField(source='get_type_display')
-    status_display = serializers.ReadOnlyField(source='get_status_display')
-    mfa_display = serializers.ReadOnlyField(source='get_mfa_display')
+    type_display = serializers.ReadOnlyField(source='get_type_display', label=_('Type for display'))
+    status_display = serializers.ReadOnlyField(source='get_status_display', label=_('Status for display'))
+    mfa_display = serializers.ReadOnlyField(source='get_mfa_display', label=_('MFA for display'))
 
     class Meta:
         model = models.UserLoginLog
         fields = (
             'id', 'username', 'type', 'type_display', 'ip', 'city', 'user_agent',
-            'mfa', 'reason', 'status', 'status_display', 'datetime', 'mfa_display'
+            'mfa', 'reason', 'status', 'status_display', 'datetime', 'mfa_display',
+            'backend'
         )
+        extra_kwargs = {
+            "user_agent": {'label': _('User agent')}
+        }
 
 
 class OperateLogSerializer(serializers.ModelSerializer):
@@ -75,13 +79,14 @@ class CommandExecutionSerializer(serializers.ModelSerializer):
             'hosts': {'label': _('Hosts')},  # 外键，会生成 sql。不在 model 上修改
             'run_as': {'label': _('Run as')},
             'user': {'label': _('User')},
+            'run_as_display': {'label': _('Run as for display')},
+            'user_display': {'label': _('User for display')},
         }
 
     @classmethod
     def setup_eager_loading(cls, queryset):
         """ Perform necessary eager loading of data. """
-        queryset = queryset.annotate(user_display=F('user__name'))\
-            .annotate(run_as_display=F('run_as__name'))
+        queryset = queryset.prefetch_related('user', 'run_as', 'hosts')
         return queryset
 
 

apps/audits/signals_handler.py

@@ -5,6 +5,8 @@ from django.db.models.signals import post_save, post_delete
 from django.dispatch import receiver
 from django.db import transaction
 from django.utils import timezone
+from django.contrib.auth import BACKEND_SESSION_KEY
+from django.utils.translation import ugettext_lazy as _
 from rest_framework.renderers import JSONRenderer
 from rest_framework.request import Request
 
@@ -32,6 +34,19 @@ MODELS_NEED_RECORD = (
 )
 
 
+LOGIN_BACKEND = {
+    'PublicKeyAuthBackend': _('SSH Key'),
+    'RadiusBackend': User.Source.radius.label,
+    'RadiusRealmBackend': User.Source.radius.label,
+    'LDAPAuthorizationBackend': User.Source.ldap.label,
+    'ModelBackend': _('Password'),
+    'SSOAuthentication': _('SSO'),
+    'CASBackend': User.Source.cas.label,
+    'OIDCAuthCodeBackend': User.Source.openid.label,
+    'OIDCAuthPasswordBackend': User.Source.openid.label,
+}
+
+
 def create_operate_log(action, sender, resource):
     user = current_request.user if current_request else None
     if not user or not user.is_authenticated:
@@ -109,6 +124,17 @@ def on_audits_log_create(sender, instance=None, **kwargs):
     sys_logger.info(msg)
 
 
+def get_login_backend(request):
+    backend = request.session.get('auth_backend', '') or request.session.get(BACKEND_SESSION_KEY, '')
+
+    backend = backend.rsplit('.', maxsplit=1)[-1]
+    if backend in LOGIN_BACKEND:
+        return LOGIN_BACKEND[backend]
+    else:
+        logger.warn(f'LOGIN_BACKEND_NOT_FOUND: {backend}')
+        return ''
+
+
 def generate_data(username, request):
     user_agent = request.META.get('HTTP_USER_AGENT', '')
     login_ip = get_request_ip(request) or '0.0.0.0'
@@ -122,7 +148,8 @@ def generate_data(username, request):
         'ip': login_ip,
         'type': login_type,
         'user_agent': user_agent,
-        'datetime': timezone.now()
+        'datetime': timezone.now(),
+        'backend': get_login_backend(request)
     }
     return data
 

apps/audits/tasks.py

@@ -2,32 +2,44 @@
 #
 import datetime
 from django.utils import timezone
-from django.conf import settings
 from celery import shared_task
 
-from ops.celery.decorator import register_as_period_task
+from ops.celery.decorator import (
+    register_as_period_task, after_app_shutdown_clean_periodic
+)
 from .models import UserLoginLog, OperateLog
+from common.utils import get_log_keep_day
 
 
-@register_as_period_task(interval=3600*24)
 @shared_task
+@after_app_shutdown_clean_periodic
 def clean_login_log_period():
     now = timezone.now()
-    try:
-        days = int(settings.LOGIN_LOG_KEEP_DAYS)
-    except ValueError:
-        days = 9999
+    days = get_log_keep_day('LOGIN_LOG_KEEP_DAYS')
     expired_day = now - datetime.timedelta(days=days)
     UserLoginLog.objects.filter(datetime__lt=expired_day).delete()
 
 
-@register_as_period_task(interval=3600*24)
 @shared_task
+@after_app_shutdown_clean_periodic
 def clean_operation_log_period():
     now = timezone.now()
-    try:
-        days = int(settings.LOGIN_LOG_KEEP_DAYS)
-    except ValueError:
-        days = 9999
+    days = get_log_keep_day('OPERATE_LOG_KEEP_DAYS')
     expired_day = now - datetime.timedelta(days=days)
     OperateLog.objects.filter(datetime__lt=expired_day).delete()
+
+
+@shared_task
+def clean_ftp_log_period():
+    now = timezone.now()
+    days = get_log_keep_day('FTP_LOG_KEEP_DAYS')
+    expired_day = now - datetime.timedelta(days=days)
+    OperateLog.objects.filter(datetime__lt=expired_day).delete()
+
+
+@register_as_period_task(interval=3600*24)
+@shared_task
+def clean_audits_log_period():
+    clean_login_log_period()
+    clean_operation_log_period()
+    clean_ftp_log_period()

apps/authentication/api/auth.py

@@ -4,7 +4,6 @@ import uuid
 
 from django.core.cache import cache
 from django.shortcuts import get_object_or_404
-from rest_framework.permissions import AllowAny
 from rest_framework.response import Response
 from rest_framework.views import APIView
 
@@ -54,12 +53,3 @@ class UserConnectionTokenApi(RootOrgViewMixin, APIView):
             return Response(value)
         else:
             return Response({'user': value['user']})
-
-    def get_permissions(self):
-        if self.request.query_params.get('user-only', None):
-            self.permission_classes = (AllowAny,)
-        return super().get_permissions()
-
-
-
-

apps/authentication/api/login_confirm.py

@@ -45,5 +45,5 @@ class TicketStatusApi(mixins.AuthMixin, APIView):
         ticket = self.get_ticket()
         if ticket:
             request.session.pop('auth_ticket_id', '')
-            ticket.perform_status('closed', request.user)
+            ticket.close(processor=request.user)
         return Response('', status=200)

apps/authentication/api/sso.py

@@ -60,6 +60,7 @@ class SSOViewSet(AuthMixin, JmsGenericViewSet):
         此接口违反了 `Restful` 的规范
         `GET` 应该是安全的方法，但此接口是不安全的
         """
+        request.META['HTTP_X_JMS_LOGIN_TYPE'] = 'W'
         authkey = request.query_params.get(AUTH_KEY)
         next_url = request.query_params.get(NEXT_URL)
         if not next_url or not next_url.startswith('/'):