fix: 完善消息队列 redis 连接未关闭

* fix: 修复 es 命令存储过滤不准确

* fix: 无效的 es 报 500

* fix: 修复索引不存在时报错

* fix: 将 es 的 doc_type 默认值改为 _doc

Co-authored-by: xinwen <coderWen@126.com>

.github/ISSUE_TEMPLATE.md

@@ -1,17 +0,0 @@
-[简述你的问题]
-
-
-##### 使用版本
-[请提供你使用的JumpServer版本 如 2.0.1 注: 1.4及以下版本不再提供支持]
-
-##### 问题复现步骤
-1. [步骤1]
-2. [步骤2]
-
-##### 具体表现[截图可能会更好些,最好能截全]
-
-
-##### 其他
-
-
-[注:] 完成后请关闭 issue

.github/ISSUE_TEMPLATE/----.md

@@ -0,0 +1,10 @@
+---
+name: 需求建议
+about: 提出针对本项目的想法和建议
+title: "[Feature] "
+labels: 类型:需求
+assignees: ibuler
+
+---
+
+**请描述您的需求或者改进建议.**

.github/ISSUE_TEMPLATE/bug---.md

@@ -0,0 +1,22 @@
+---
+name: Bug 提交
+about: 提交产品缺陷帮助我们更好的改进
+title: "[Bug] "
+labels: 类型:bug
+assignees: wojiushixiaobai
+
+---
+
+**JumpServer 版本(v1.5.9以下不再支持)**
+
+
+**浏览器版本**
+
+
+**Bug 描述**
+
+
+**Bug 重现步骤(有截图更好)**
+1.  
+2. 
+3. 

.github/ISSUE_TEMPLATE/question.md

@@ -0,0 +1,10 @@
+---
+name: 问题咨询
+about: 提出针对本项目安装部署、使用及其他方面的相关问题
+title: "[Question] "
+labels: 类型:提问
+assignees: wojiushixiaobai
+
+---
+
+**请描述您的问题.**

.github/workflows/jms-generic-action-handler.yml

@@ -0,0 +1,12 @@
+on: [push, pull_request, release]
+
+name: JumpServer repos generic handler
+
+jobs:
+  generic_handler:
+    name: Run generic handler
+    runs-on: ubuntu-latest
+    steps:
+      - uses: jumpserver/action-generic-handler@master
+        env:
+          GITHUB_TOKEN: ${{ secrets.PRIVATE_TOKEN }}

.gitignore

@@ -15,6 +15,7 @@ dump.rdb
 .tox
 .cache/
 .idea/
+.vscode/
 db.sqlite3
 config.py
 config.yml

Dockerfile

@@ -1,5 +1,6 @@
-FROM registry.fit2cloud.com/public/python:v3 as stage-build
-MAINTAINER Jumpserver Team <ibuler@qq.com>
+# 编译代码
+FROM python:3.8.6-slim as stage-build
+MAINTAINER JumpServer Team <ibuler@qq.com>
 ARG VERSION
 ENV VERSION=$VERSION
 
@@ -8,27 +9,39 @@ ADD . .
 RUN cd utils && bash -ixeu build.sh
 
 
-FROM registry.fit2cloud.com/public/python:v3
+# 构建运行时环境
+FROM python:3.8.6-slim
+ARG PIP_MIRROR=https://pypi.douban.com/simple
+ENV PIP_MIRROR=$PIP_MIRROR
+ARG PIP_JMS_MIRROR=https://pypi.douban.com/simple
+ENV PIP_JMS_MIRROR=$PIP_JMS_MIRROR
+
 WORKDIR /opt/jumpserver
+
+COPY ./requirements/deb_buster_requirements.txt ./requirements/deb_buster_requirements.txt
+RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list \
+    && sed -i 's/security.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list \
+    && apt update \
+    && grep -v '^#' ./requirements/deb_buster_requirements.txt | xargs apt -y install \
+    && rm -rf /var/lib/apt/lists/* \
+    && localedef -c -f UTF-8 -i zh_CN zh_CN.UTF-8 \
+    && cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
+
+COPY ./requirements/requirements.txt ./requirements/requirements.txt
+RUN pip install --upgrade pip==20.2.4 setuptools==49.6.0 wheel==0.34.2 -i ${PIP_MIRROR} \
+    && pip config set global.index-url ${PIP_MIRROR} \
+    && pip install --no-cache-dir $(grep 'jms' requirements/requirements.txt) -i ${PIP_JMS_MIRROR} \
+    && pip install --no-cache-dir -r requirements/requirements.txt
+
 COPY --from=stage-build /opt/jumpserver/release/jumpserver /opt/jumpserver
-
-RUN useradd jumpserver
-
-RUN yum -y install epel-release && \
-      echo -e "[mysql]\nname=mysql\nbaseurl=https://mirrors.tuna.tsinghua.edu.cn/mysql/yum/mysql57-community-el6/\ngpgcheck=0\nenabled=1" > /etc/yum.repos.d/mysql.repo
-
-COPY . .
-RUN yum -y install $(cat requirements/rpm_requirements.txt)
-RUN pip install --upgrade pip setuptools && pip install wheel && \
-    pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements/requirements.txt || pip install -r requirements/requirements.txt
-RUN mkdir -p /root/.ssh/ && echo -e "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null" > /root/.ssh/config
+RUN mkdir -p /root/.ssh/ \
+    && echo "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null" > /root/.ssh/config
 
 RUN echo > config.yml
 VOLUME /opt/jumpserver/data
 VOLUME /opt/jumpserver/logs
 
 ENV LANG=zh_CN.UTF-8
-ENV LC_ALL=zh_CN.UTF-8
 
 EXPOSE 8070
 EXPOSE 8080

README.md

@@ -1,9 +1,18 @@
 # JumpServer 多云环境下更好用的堡垒机
 
-[![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
-[![Django](https://img.shields.io/badge/django-2.2-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
+[![License](https://shields.io/github/license/jumpserver/jumpserver)](https://www.gnu.org/licenses/old-licenses/gpl-2.0.html)
+[![Release Downloads](https://shields.io/github/downloads/jumpserver/jumpserver/total)](https://github.com/jumpserver/jumpserver/releases)
 [![Docker Pulls](https://img.shields.io/docker/pulls/jumpserver/jms_all.svg)](https://hub.docker.com/u/jumpserver)
 
+- [ENGLISH](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
+
+
+|《新一代堡垒机建设指南》开放下载|
+|------------------|
+|本白皮书由JumpServer开源项目组编著而成。编写团队从企业实践和技术演进的双重视角出发，结合自身在身份与访问安全领域长期研发及落地经验组织撰写，同时积极听取行业内专家的意见和建议，在此基础上完成了本白皮书的编写任务。下载链接：https://jinshuju.net/f/E0qAl8|
+
+--------------------------
+
 JumpServer 是全球首款开源的堡垒机，使用 GNU GPL v2.0 开源协议，是符合 4A 规范的运维安全审计系统。
 
 JumpServer 使用 Python / Django 为主进行开发，遵循 Web 2.0 规范，配备了业界领先的 Web Terminal 方案，交互界面美观、用户体验好。
@@ -12,39 +21,24 @@ JumpServer 采纳分布式架构，支持多机房跨区域部署，支持横向
 
 改变世界，从一点点开始。
 
-> 注: [KubeOperator](https://github.com/KubeOperator/KubeOperator) 是 JumpServer 团队在 Kubernetes 领域的的又一全新力作，欢迎关注和使用。
 
 ## 特色优势
 
-- 开源: 零门槛，线上快速获取和安装, 修复版本视情况而定；
-, 修复版本视情况而定- 分布式: 轻松支持大规模并发访问；
+- 开源: 零门槛，线上快速获取和安装；
+- 分布式: 轻松支持大规模并发访问；
 - 无插件: 仅需浏览器，极致的 Web Terminal 使用体验；
 - 多云支持: 一套系统，同时管理不同云上面的资产；
 - 云端存储: 审计录像云端存储，永不丢失；
-- 多租户: 一套系统，多个子公司和部门同时使用。
+- 多租户: 一套系统，多个子公司和部门同时使用；
+- 多应用支持: 数据库，Windows远程应用，Kubernetes。
 
-## 版本说明
-
-自 v2.0.0 发布后， JumpServer 版本号命名将变更为：v大版本.功能版本.Bug修复版本。比如：
-
-```
-v2.0.1 是 v2.0.0 之后的Bug修复版本；
-v2.1.0 是 v2.0.0 之后的功能版本。
-```
-
-像其它优秀开源项目一样，JumpServer 每个月会发布一个功能版本，并同时维护 3 个功能版本。比如：
-
-```
-在 v2.4 发布前，我们会同时维护 v2.1、v2.2、v2.3；
-在 v2.4 发布后，我们会同时维护 v2.2、v2.3、v2.4；v2.1 会停止维护。
-```
 
 ## 功能列表
 
 <table>
   <tr>
-    <td rowspan="8">身份认证<br>Authentication</td>
-    <td rowspan="5">登录认证</td>
+    <td rowspan="11">身份认证<br>Authentication</td>
+    <td rowspan="7">登录认证</td>
     <td>资源统一登录与认证</td>
   </tr>
   <tr>
@@ -59,6 +53,12 @@ v2.1.0 是 v2.0.0 之后的功能版本。
   <tr>
     <td>CAS 认证 （实现单点登录）</td>
   </tr>
+  <tr>
+    <td>钉钉认证 （扫码登录）</td>
+  </tr>
+  <tr>
+    <td>企业微信认证 （扫码登录）</td>
+  </tr>
   <tr>
     <td rowspan="2">MFA认证</td>
     <td>MFA 二次认证（Google Authenticator）</td>
@@ -67,8 +67,12 @@ v2.1.0 是 v2.0.0 之后的功能版本。
     <td>RADIUS 二次认证</td>
   </tr>
   <tr>
-    <td>登录复核（X-PACK）</td>
-    <td>用户登录行为受管理员的监管与控制</td>
+    <td>登录复核</td>
+    <td>用户登录行为受管理员的监管与控制:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>登录限制</td>
+    <td>用户登录来源 IP 受管理员控制（支持黑/白名单）</td>
   </tr>
   <tr>
     <td rowspan="11">账号管理<br>Account</td>
@@ -92,26 +96,26 @@ v2.1.0 是 v2.0.0 之后的功能版本。
     <td>密码过期设置</td>
   </tr>
   <tr>
-    <td rowspan="2">批量改密（X-PACK）</td>
-    <td>定期批量改密</td>
+    <td rowspan="2">批量改密</td>
+    <td>定期批量改密:small_orange_diamond:</td>
   </tr>
   <tr>
-    <td>多种密码策略</td>
+    <td>多种密码策略:small_orange_diamond:</td>
   </tr>
   <tr>
-    <td>多云纳管（X-PACK）</td>
-    <td>对私有云、公有云资产自动统一纳管</td>
+    <td>多云纳管 </td>
+    <td>对私有云、公有云资产自动统一纳管:small_orange_diamond:</td>
   </tr>
   <tr>
-    <td>收集用户（X-PACK）</td>
-    <td>自定义任务定期收集主机用户</td>
+    <td>收集用户 </td>
+    <td>自定义任务定期收集主机用户:small_orange_diamond:</td>
   </tr>
   <tr>
-    <td>密码匣子（X-PACK）</td>
-    <td>统一对资产主机的用户密码进行查看、更新、测试操作</td>
+    <td>密码匣子 </td>
+    <td>统一对资产主机的用户密码进行查看、更新、测试操作:small_orange_diamond:</td>
   </tr>
   <tr>
-    <td rowspan="15">授权控制<br>Authorization</td>
+    <td rowspan="17">授权控制<br>Authorization</td>
     <td>多维授权</td>
     <td>对用户、用户组、资产、资产节点、应用以及系统用户进行授权</td>
   </tr>
@@ -133,7 +137,7 @@ v2.1.0 是 v2.0.0 之后的功能版本。
     <td>实现更细粒度的应用级授权</td>
   </tr>
   <tr>
-    <td>MySQL 数据库应用、RemoteApp 远程应用（X-PACK）</td>
+    <td>MySQL 数据库应用、RemoteApp 远程应用:small_orange_diamond: </td>
   </tr>
   <tr>
     <td>动作授权</td>
@@ -160,20 +164,30 @@ v2.1.0 是 v2.0.0 之后的功能版本。
     <td>实现 Web SFTP 文件管理</td>
   </tr>
   <tr>
-    <td>工单管理（X-PACK）</td>
-    <td>支持对用户登录请求行为进行控制</td>
+    <td>工单管理</td>
+    <td>支持对用户登录请求行为进行控制:small_orange_diamond:</td>
   </tr>
   <tr>
-    <td>组织管理（X-PACK）</td>
-    <td>实现多租户管理与权限隔离</td>
+    <td rowspan="2">访问控制</td>
+    <td>登录资产复核（通过 SSH/Telnet 协议登录资产）:small_orange_diamond:</td>
   </tr>
   <tr>
-    <td rowspan="7">安全审计<br>Audit</td>
+    <td>命令执行复核:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>组织管理</td>
+    <td>实现多租户管理与权限隔离:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td rowspan="8">安全审计<br>Audit</td>
     <td>操作审计</td>
     <td>用户操作行为审计</td>
   </tr>
   <tr>
-    <td rowspan="2">会话审计</td>
+    <td rowspan="3">会话审计</td>
+    <td>在线会话内容监控</td>
+  </tr>
+  <tr>
     <td>在线会话内容审计</td>
   </tr>
   <tr>
@@ -184,7 +198,7 @@ v2.1.0 是 v2.0.0 之后的功能版本。
     <td>支持对 Linux、Windows 等资产操作的录像进行回放审计</td>
   </tr>
   <tr>
-    <td>支持对 RemoteApp（X-PACK）、MySQL 等应用操作的录像进行回放审计</td>
+    <td>支持对 RemoteApp:small_orange_diamond:、MySQL 等应用操作的录像进行回放审计</td>
   </tr>
   <tr>
     <td>指令审计</td>
@@ -194,13 +208,88 @@ v2.1.0 是 v2.0.0 之后的功能版本。
     <td>文件传输</td>
     <td>可对文件的上传、下载记录进行审计</td>
   </tr>
+  <tr>
+    <td rowspan="20">数据库审计<br>Database</td>
+    <td rowspan="2">连接方式</td>
+    <td>命令方式</td>
+  </tr>
+  <tr>
+    <td>Web UI方式 :small_orange_diamond:</td>
+  </tr>
+
+  <tr>
+    <td rowspan="4">支持的数据库</td>
+    <td>MySQL</td>
+  </tr>
+  <tr>
+    <td>Oracle :small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>MariaDB :small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>PostgreSQL :small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td rowspan="6">功能亮点</td>
+    <td>语法高亮</td>
+  </tr>
+  <tr>
+    <td>SQL格式化</td>
+  </tr>
+  <tr>
+    <td>支持快捷键</td>
+  </tr>
+  <tr>
+    <td>支持选中执行</td>
+  </tr>
+  <tr>
+    <td>SQL历史查询</td>
+  </tr>
+  <tr>
+    <td>支持页面创建 DB, TABLE</td>
+  </tr>
+  <tr>
+    <td rowspan="2">会话审计</td>
+    <td>命令记录</td>
+  </tr>
+  <tr>
+    <td>录像回放</td>
+  </tr>
 </table>
 
+**说明**: 带 :small_orange_diamond: 后缀的是 X-PACK 插件有的功能
+
 ## 快速开始
 
 - [极速安装](https://docs.jumpserver.org/zh/master/install/setup_by_fast/)
 - [完整文档](https://docs.jumpserver.org)
-- [演示视频](https://jumpserver.oss-cn-hangzhou.aliyuncs.com/jms-media/%E3%80%90%E6%BC%94%E7%A4%BA%E8%A7%86%E9%A2%91%E3%80%91Jumpserver%20%E5%A0%A1%E5%9E%92%E6%9C%BA%20V1.5.0%20%E6%BC%94%E7%A4%BA%E8%A7%86%E9%A2%91%20-%20final.mp4)
+- [演示视频](https://www.bilibili.com/video/BV1ZV41127GB)
+- [手动安装](https://github.com/jumpserver/installer)
+
+## 组件项目
+- [Lina](https://github.com/jumpserver/lina) JumpServer Web UI 项目
+- [Luna](https://github.com/jumpserver/luna) JumpServer Web Terminal 项目
+- [KoKo](https://github.com/jumpserver/koko) JumpServer 字符协议 Connector 项目，替代原来 Python 版本的 [Coco](https://github.com/jumpserver/coco)
+- [Lion](https://github.com/jumpserver/lion-release) JumpServer 图形协议 Connector 项目，依赖 [Apache Guacamole](https://guacamole.apache.org/)
+
+## 贡献
+如果有你好的想法创意，或者帮助我们修复了 Bug, 欢迎提交 Pull Request
+
+感谢以下贡献者，让 JumpServer 更加完善
+
+<a href="https://github.com/jumpserver/jumpserver/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=jumpserver/jumpserver" />
+</a>
+
+
+## 致谢
+- [Apache Guacamole](https://guacamole.apache.org/) Web页面连接 RDP, SSH, VNC协议设备，JumpServer 图形化组件 Lion 依赖 
+- [OmniDB](https://omnidb.org/) Web页面连接使用数据库，JumpServer Web数据库依赖
+
+
+## JumpServer 企业版 
+- [申请企业版试用](https://jinshuju.net/f/kyOYpi)
 
 ## 案例研究
 
@@ -232,3 +321,4 @@ Licensed under The GNU General Public License version 2 (GPLv2)  (the "License")
 https://www.gnu.org/licenses/gpl-2.0.html
 
 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
+

README_EN.md

@@ -1,30 +1,245 @@
-## Jumpserver 
+# Jumpserver - The Bastion Host for Multi-Cloud Environment
 
-![Total visitor](https://visitor-count-badge.herokuapp.com/total.svg?repo_id=jumpserver)
-![Visitors in today](https://visitor-count-badge.herokuapp.com/today.svg?repo_id=jumpserver)
 [![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
-[![Django](https://img.shields.io/badge/django-2.1-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
-[![Ansible](https://img.shields.io/badge/ansible-2.4.2.0-blue.svg?style=plastic)](https://www.ansible.com/)
-[![Paramiko](https://img.shields.io/badge/paramiko-2.4.1-green.svg?style=plastic)](http://www.paramiko.org/)
+[![Django](https://img.shields.io/badge/django-2.2-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
+[![Docker Pulls](https://img.shields.io/docker/pulls/jumpserver/jms_all.svg)](https://hub.docker.com/u/jumpserver)
 
+- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README.md)
+
+|![notification](https://raw.githubusercontent.com/goharbor/website/master/docs/img/readme/bell-outline-badged.svg)Security Notice|
+|------------------|
+|On 15th January 2021, JumpServer found a critical bug for remote execution vulnerability. Please fix it asap! [For more detail](https://github.com/jumpserver/jumpserver/issues/5533) Thanks for **reactivity of Alibaba Hackerone bug bounty program** report use the bug|
+
+--------------------------
+
+Jumpserver is the world's first open-source Bastion Host and is licensed under the GNU GPL v2.0. It is a 4A-compliant professional operation and maintenance security audit system.
+
+Jumpserver uses Python / Django for development, follows Web 2.0 specifications, and is equipped with an industry-leading Web Terminal solution that provides a beautiful user interface and great user experience
+
+Jumpserver adopts a distributed architecture to support multi-branch deployment across multiple cross-regional areas. The central node provides APIs, and login nodes are deployed in each branch. It can be scaled horizontally without concurrency restrictions.
+
+Change the world by taking every little step
 
 ----
+### Advantages
 
-- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
+- Open Source: huge transparency and free to access with quick installation process.
+- Distributed: support large-scale concurrent access with ease.
+- No Plugin required: all you need is a browser, the ultimate Web Terminal experience.
+- Multi-Cloud supported: a unified system to manage assets on different clouds at the same time
+- Cloud storage: audit records are stored in the cloud. Data lost no more!
+- Multi-Tenant system: multiple subsidiary companies or departments access the same system simultaneously.
+- Many applications supported: link to databases, windows remote applications, and Kubernetes cluster, etc.
 
-Jumpserver is the first fully open source bastion in the world, based on the GNU GPL v2.0 open source protocol. Jumpserver is a  professional operation and maintenance audit system conforms to 4A specifications.
+## Features List
 
-Jumpserver is developed using Python / Django, conforms to the Web 2.0 specification, and is equipped with the industry-leading Web Terminal solution which have beautiful interface and great user experience.
+<table>
+  <tr>
+    <td rowspan="8">Authentication</td>
+    <td rowspan="5">Login</td>
+    <td>Unified way to access and authenticate resources</td>
+  </tr>
+  <tr>
+    <td>LDAP/AD Authentication</td>
+  </tr>
+  <tr>
+    <td>RADIUS Authentication</td>
+  </tr>
+  <tr>
+    <td>OpenID Authentication（Single Sign-On）</td>
+  </tr>
+  <tr>
+    <td>CAS Authentication （Single Sign-On）</td>
+  </tr>
+  <tr>
+    <td rowspan="2">MFA (Multi-Factor Authentication)</td>
+    <td>Use Google Authenticator for MFA</td>
+  </tr>
+  <tr>
+    <td>RADIUS (Remote Authentication Dial In User Service)</td>
+  </tr>
+  <tr>
+    <td>Login Supervision</td>
+    <td>Any user’s login behavior is supervised and controlled by the administrator:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td rowspan="11">Accounting</td>
+    <td rowspan="2">Centralized Accounts Management</td>
+    <td>Admin Users management</td>
+  </tr>
+  <tr>
+    <td>System Users management</td>
+  </tr>
+  <tr>
+    <td rowspan="4">Unified Password Management</td>
+    <td>Asset password custody (a matrix storing all asset password with dense security)</td>
+  </tr>
+  <tr>
+    <td>Auto-generated passwords</td>
+  </tr>
+  <tr>
+    <td>Automatic password handling (auto login assets)</td>
+  </tr>
+  <tr>
+    <td>Password expiration settings</td>
+  </tr>
+  <tr>
+    <td rowspan="2">Password change Schedular</td>
+    <td>Support regular batch Linux/Windows assets password changing:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>Implement multiple password strategies:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>Multi-Cloud Management</td>
+    <td>Automatically manage private cloud and public cloud assets in a unified platform :small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>Users Acquisition </td>
+    <td>Create regular custom tasks to collect system users in selected assets to identify and track the privileges ownership:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>Password Vault </td>
+    <td>Unified operations to check, update, and test system user password to prevent stealing or unauthorised sharing of passwords:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td rowspan="15">Authorization</td>
+    <td>Multi-Dimensional</td>
+    <td>Granting users or user groups to access assets, asset nodes, or applications through system users. Providing precise access control to different roles of users</td>
+  </tr>
+  <tr>
+    <td rowspan="4">Assets</td>
+    <td>Assets are arranged and displayed in a tree structure </td>
+  </tr>
+  <tr>
+    <td>Assets and Nodes have immense flexibility for authorizing</td>
+  </tr>
+  <tr>
+    <td>Assets in nodes inherit authorization automatically</td>
+  </tr>
+  <tr>
+    <td>child nodes automatically inherit authorization from parent nodes</td>
+  </tr>
+  <tr>
+    <td rowspan="2">Application</td>
+    <td>Provides granular access control for privileged users on application level to protect from unauthorized access and unintentional errors</td>
+  </tr>
+  <tr>
+    <td>Database applications (MySQL, Oracle, PostgreSQL, MariaDB, etc.) and Remote App:small_orange_diamond: </td>
+  </tr>
+  <tr>
+    <td>Actions</td>
+    <td>Deeper restriction on the control of file upload, download and connection actions of authorized assets. Control the permission of clipboard copy/paste (from outer terminal to current asset)</td>
+  </tr>
+  <tr>
+    <td>Time Bound</td>
+    <td>Sharply limited the available (accessible) time for account access to the authorized resources to reduce the risk and attack surface drastically</td>
+  </tr>
+  <tr>
+    <td>Privileged Assignment</td>
+    <td>Assign the denied/allowed command lists to different system users as privilege elevation, with the latter taking the form of allowing particular commands to be run with a higher level of privileges. (Minimize insider threat)</td>
+  </tr>
+  <tr>
+    <td>Command Filtering</td>
+    <td>Creating list of restriction commands that you would like to assign to different authorized system users for filtering purpose</td>
+  </tr>
+  <tr>
+    <td>File Transfer and Management</td>
+    <td>Support SFTP file upload/download</td>
+  </tr>
+  <tr>
+    <td>File Management</td>
+    <td>Provide a Web UI for SFTP file management</td>
+  </tr>
+  <tr>
+    <td>Workflow Management</td>
+    <td>Manage user login confirmation requests and assets or applications authorization requests for Just-In-Time Privileges functionality:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>Group Management </td>
+    <td>Establishing a multi-tenant ecosystem that able authority isolation to keep malicious actors away from sensitive administrative backends:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td rowspan="8">Auditing</td>
+    <td>Operations</td>
+    <td>Auditing user operation behaviors for any access or usage of given privileged accounts</td>
+  </tr>
+  <tr>
+    <td rowspan="2">Session</td>
+    <td>Support real-time session audit</td>
+  </tr>
+  <tr>
+    <td>Full history of all previous session audits</td>
+  </tr>
+  <tr>
+    <td rowspan="3">Video</td>
+    <td>Complete session audit and playback recordings on assets operation (Linux, Windows)</td>
+  </tr>
+  <tr>
+    <td>Full recordings of RemoteApp, MySQL, and Kubernetes:small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>Supports uploading recordings to public clouds</td>
+  </tr>
+  <tr>
+    <td>Command</td>
+    <td>Command auditing on assets and applications operation. Send warning alerts when executing illegal commands</td>
+  </tr>
+  <tr>
+    <td>File Transfer</td>
+    <td>Full recordings of file upload and download</td>
+  </tr>
+  <tr>
+    <td rowspan="20">Database</td>
+    <td rowspan="2">How to connect</td>
+    <td>Command line</td>
+  </tr>
+  <tr>
+    <td>Built-in Web UI:small_orange_diamond:</td>
+  </tr>
 
-Jumpserver adopts a distributed architecture to support multi-branch deployment across multiple areas. The central node provides APIs, and login nodes are deployed in each branch. It can be scaled horizontally without concurrency restrictions.
+  <tr>
+    <td rowspan="4">Supported Database</td>
+    <td>MySQL</td>
+  </tr>
+  <tr>
+    <td>Oracle :small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>MariaDB :small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td>PostgreSQL :small_orange_diamond:</td>
+  </tr>
+  <tr>
+    <td rowspan="6">Feature Highlights</td>
+    <td>Syntax highlights</td>
+  </tr>
+  <tr>
+    <td>Prettier SQL formmating</td>
+  </tr>
+  <tr>
+    <td>Support Shortcuts</td>
+  </tr>
+  <tr>
+    <td>Support selected SQL statements</td>
+  </tr>
+  <tr>
+    <td>SQL commands history query</td>
+  </tr>
+  <tr>
+    <td>Support page creation: DB, TABLE</td>
+  </tr>
+  <tr>
+    <td rowspan="2">Session Auditing</td>
+    <td>Full records of command</td>
+  </tr>
+  <tr>
+    <td>Playback videos</td>
+  </tr>
+</table>
 
-Change the world, starting from little things.
-
-----
-
-### Features
-
- ![Jumpserver 功能](https://jumpserver-release.oss-cn-hangzhou.aliyuncs.com/Jumpserver148.jpeg "Jumpserver 功能")
+**Note**: Rows with :small_orange_diamond: at the end of the sentence means that it is X-PACK features exclusive ([Apply for X-PACK Trial](https://jinshuju.net/f/kyOYpi))
 
 ### Start 
 
@@ -47,8 +262,52 @@ We provide online demo, demo video and screenshots to get you started quickly.
 We provide the SDK for your other systems to quickly interact with the Jumpserver API.
 
 - [Python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver other components use this SDK to complete the interaction.
-- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) 恺珺同学提供的Java版本的SDK thanks to 恺珺 for provide Java SDK
+- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) Thanks to 恺珺 for providing his Java SDK vesrion.
 
+## JumpServer Component Projects
+- [Lina](https://github.com/jumpserver/lina) JumpServer Web UI
+- [Luna](https://github.com/jumpserver/luna) JumpServer Web Terminal
+- [KoKo](https://github.com/jumpserver/koko) JumpServer Character protocaol Connector, replace original Python Version [Coco](https://github.com/jumpserver/coco)
+- [Guacamole](https://github.com/jumpserver/docker-guacamole) JumpServer Graphics protocol Connector，rely on [Apache Guacamole](https://guacamole.apache.org/)
+
+## Contribution
+If you have any good ideas or helping us to fix bugs, please submit a Pull Request and accept our thanks :)
+
+Thanks to the following contributors for making JumpServer better everyday!
+
+<a href="https://github.com/jumpserver/jumpserver/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=jumpserver/jumpserver" />
+</a>
+
+
+## Thanks to
+- [Apache Guacamole](https://guacamole.apache.org/) Web page connection RDP, SSH, VNC protocol equipment. JumpServer graphical connection dependent.
+- [OmniDB](https://omnidb.org/) Web page connection to databases. JumpServer Web database dependent.
+
+
+## JumpServer Enterprise Version 
+- [Apply for it](https://jinshuju.net/f/kyOYpi)
+
+## Case Study
+
+- [JumpServer 堡垒机护航顺丰科技超大规模资产安全运维](https://blog.fit2cloud.com/?p=1147)；
+- [JumpServer 堡垒机让“大智慧”的混合 IT 运维更智慧](https://blog.fit2cloud.com/?p=882)；
+- [携程 JumpServer 堡垒机部署与运营实战](https://blog.fit2cloud.com/?p=851)；
+- [小红书的JumpServer堡垒机大规模资产跨版本迁移之路](https://blog.fit2cloud.com/?p=516)；
+- [JumpServer堡垒机助力中手游提升多云环境下安全运维能力](https://blog.fit2cloud.com/?p=732)；
+- [中通快递：JumpServer主机安全运维实践](https://blog.fit2cloud.com/?p=708)；
+- [东方明珠：JumpServer高效管控异构化、分布式云端资产](https://blog.fit2cloud.com/?p=687)；
+- [江苏农信：JumpServer堡垒机助力行业云安全运维](https://blog.fit2cloud.com/?p=666)。
+
+## For safety instructions
+
+JumpServer is a security product. Please refer to [Basic Security Recommendations](https://docs.jumpserver.org/zh/master/install/install_security/) for deployment and installation.
+
+If you find a security problem, please contact us directly：
+
+- ibuler@fit2cloud.com 
+- support@fit2cloud.com 
+- 400-052-0755
 
 ### License & Copyright
 Copyright (c) 2014-2019 Beijing Duizhan Tech, Inc., All rights reserved.

apps/.gitattributes

@@ -0,0 +1,2 @@
+*.js linguist-language=python
+*.html linguist-language=python

apps/acls/admin.py

@@ -0,0 +1,3 @@
+from django.contrib import admin
+
+# Register your models here.

apps/acls/api/__init__.py

@@ -0,0 +1,3 @@
+from .login_acl import *
+from .login_asset_acl import *
+from .login_asset_check import *

apps/acls/api/login_acl.py

@@ -0,0 +1,19 @@
+from common.permissions import IsOrgAdmin, HasQueryParamsUserAndIsCurrentOrgMember
+from common.drf.api import JMSBulkModelViewSet
+from ..models import LoginACL
+from .. import serializers
+
+__all__ = ['LoginACLViewSet', ]
+
+
+class LoginACLViewSet(JMSBulkModelViewSet):
+    queryset = LoginACL.objects.all()
+    filterset_fields = ('name', 'user', )
+    search_fields = filterset_fields
+    permission_classes = (IsOrgAdmin, )
+    serializer_class = serializers.LoginACLSerializer
+
+    def get_permissions(self):
+        if self.action in ["retrieve", "list"]:
+            self.permission_classes = (IsOrgAdmin, HasQueryParamsUserAndIsCurrentOrgMember)
+        return super().get_permissions()

apps/acls/api/login_asset_acl.py

@@ -0,0 +1,15 @@
+
+from orgs.mixins.api import OrgBulkModelViewSet
+from common.permissions import IsOrgAdmin
+from .. import models, serializers
+
+
+__all__ = ['LoginAssetACLViewSet']
+
+
+class LoginAssetACLViewSet(OrgBulkModelViewSet):
+    model = models.LoginAssetACL
+    filterset_fields = ('name', )
+    search_fields = filterset_fields
+    permission_classes = (IsOrgAdmin, )
+    serializer_class = serializers.LoginAssetACLSerializer

apps/acls/api/login_asset_check.py

@@ -0,0 +1,77 @@
+from django.shortcuts import get_object_or_404
+from rest_framework.response import Response
+from rest_framework.generics import CreateAPIView, RetrieveDestroyAPIView
+
+from common.permissions import IsAppUser
+from common.utils import reverse, lazyproperty
+from orgs.utils import tmp_to_org, tmp_to_root_org
+from tickets.api import GenericTicketStatusRetrieveCloseAPI
+from ..models import LoginAssetACL
+from .. import serializers
+
+
+__all__ = ['LoginAssetCheckAPI', 'LoginAssetConfirmStatusAPI']
+
+
+class LoginAssetCheckAPI(CreateAPIView):
+    permission_classes = (IsAppUser, )
+    serializer_class = serializers.LoginAssetCheckSerializer
+
+    def create(self, request, *args, **kwargs):
+        is_need_confirm, response_data = self.check_if_need_confirm()
+        return Response(data=response_data, status=200)
+
+    def check_if_need_confirm(self):
+        queries = {
+            'user': self.serializer.user, 'asset': self.serializer.asset,
+            'system_user': self.serializer.system_user,
+            'action': LoginAssetACL.ActionChoices.login_confirm
+        }
+        with tmp_to_org(self.serializer.org):
+            acl = LoginAssetACL.filter(**queries).valid().first()
+
+        if not acl:
+            is_need_confirm = False
+            response_data = {}
+        else:
+            is_need_confirm = True
+            response_data = self._get_response_data_of_need_confirm(acl)
+        response_data['need_confirm'] = is_need_confirm
+        return is_need_confirm, response_data
+
+    def _get_response_data_of_need_confirm(self, acl):
+        ticket = LoginAssetACL.create_login_asset_confirm_ticket(
+            user=self.serializer.user,
+            asset=self.serializer.asset,
+            system_user=self.serializer.system_user,
+            assignees=acl.reviewers.all(),
+            org_id=self.serializer.org.id
+        )
+        confirm_status_url = reverse(
+            view_name='api-acls:login-asset-confirm-status',
+            kwargs={'pk': str(ticket.id)}
+        )
+        ticket_detail_url = reverse(
+            view_name='api-tickets:ticket-detail',
+            kwargs={'pk': str(ticket.id)},
+            external=True, api_to_ui=True
+        )
+        ticket_detail_url = '{url}?type={type}'.format(url=ticket_detail_url, type=ticket.type)
+        data = {
+            'check_confirm_status': {'method': 'GET', 'url': confirm_status_url},
+            'close_confirm': {'method': 'DELETE', 'url': confirm_status_url},
+            'ticket_detail_url': ticket_detail_url,
+            'reviewers': [str(user) for user in ticket.assignees.all()],
+        }
+        return data
+
+    @lazyproperty
+    def serializer(self):
+        serializer = self.get_serializer(data=self.request.data)
+        serializer.is_valid(raise_exception=True)
+        return serializer
+
+
+class LoginAssetConfirmStatusAPI(GenericTicketStatusRetrieveCloseAPI):
+    pass
+

apps/acls/apps.py

@@ -0,0 +1,5 @@
+from django.apps import AppConfig
+
+
+class AclsConfig(AppConfig):
+    name = 'acls'

apps/acls/migrations/0001_initial.py

@@ -0,0 +1,61 @@
+# Generated by Django 3.1 on 2021-03-11 09:53
+
+from django.conf import settings
+import django.core.validators
+from django.db import migrations, models
+import django.db.models.deletion
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='LoginACL',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('created_by', models.CharField(blank=True, max_length=32, null=True, verbose_name='Created by')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+                ('date_updated', models.DateTimeField(auto_now=True, verbose_name='Date updated')),
+                ('name', models.CharField(max_length=128, verbose_name='Name')),
+                ('priority', models.IntegerField(default=50, help_text='1-100, the lower the value will be match first', validators=[django.core.validators.MinValueValidator(1), django.core.validators.MaxValueValidator(100)], verbose_name='Priority')),
+                ('is_active', models.BooleanField(default=True, verbose_name='Active')),
+                ('comment', models.TextField(blank=True, default='', verbose_name='Comment')),
+                ('ip_group', models.JSONField(default=list, verbose_name='Login IP')),
+                ('action', models.CharField(choices=[('reject', 'Reject'), ('allow', 'Allow')], default='reject', max_length=64, verbose_name='Action')),
+                ('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='login_acls', to=settings.AUTH_USER_MODEL, verbose_name='User')),
+            ],
+            options={
+                'ordering': ('priority', '-date_updated', 'name'),
+            },
+        ),
+        migrations.CreateModel(
+            name='LoginAssetACL',
+            fields=[
+                ('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('created_by', models.CharField(blank=True, max_length=32, null=True, verbose_name='Created by')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+                ('date_updated', models.DateTimeField(auto_now=True, verbose_name='Date updated')),
+                ('name', models.CharField(max_length=128, verbose_name='Name')),
+                ('priority', models.IntegerField(default=50, help_text='1-100, the lower the value will be match first', validators=[django.core.validators.MinValueValidator(1), django.core.validators.MaxValueValidator(100)], verbose_name='Priority')),
+                ('is_active', models.BooleanField(default=True, verbose_name='Active')),
+                ('comment', models.TextField(blank=True, default='', verbose_name='Comment')),
+                ('users', models.JSONField(verbose_name='User')),
+                ('system_users', models.JSONField(verbose_name='System User')),
+                ('assets', models.JSONField(verbose_name='Asset')),
+                ('action', models.CharField(choices=[('login_confirm', 'Login confirm')], default='login_confirm', max_length=64, verbose_name='Action')),
+                ('reviewers', models.ManyToManyField(blank=True, related_name='review_login_asset_acls', to=settings.AUTH_USER_MODEL, verbose_name='Reviewers')),
+            ],
+            options={
+                'ordering': ('priority', '-date_updated', 'name'),
+                'unique_together': {('name', 'org_id')},
+            },
+        ),
+    ]

apps/acls/models/__init__.py

@@ -0,0 +1,2 @@
+from .login_acl import *
+from .login_asset_acl import *

apps/acls/models/base.py

@@ -0,0 +1,35 @@
+from django.db import models
+from django.utils.translation import ugettext_lazy as _
+from django.core.validators import MinValueValidator, MaxValueValidator
+from common.mixins import CommonModelMixin
+
+
+__all__ = ['BaseACL', 'BaseACLQuerySet']
+
+
+class BaseACLQuerySet(models.QuerySet):
+    def active(self):
+        return self.filter(is_active=True)
+
+    def inactive(self):
+        return self.filter(is_active=False)
+
+    def valid(self):
+        return self.active()
+
+    def invalid(self):
+        return self.inactive()
+
+
+class BaseACL(CommonModelMixin):
+    name = models.CharField(max_length=128, verbose_name=_('Name'))
+    priority = models.IntegerField(
+        default=50, verbose_name=_("Priority"),
+        help_text=_("1-100, the lower the value will be match first"),
+        validators=[MinValueValidator(1), MaxValueValidator(100)]
+    )
+    is_active = models.BooleanField(default=True, verbose_name=_("Active"))
+    comment = models.TextField(default='', blank=True, verbose_name=_('Comment'))
+
+    class Meta:
+        abstract = True

apps/acls/models/login_acl.py

@@ -0,0 +1,57 @@
+
+from django.db import models
+from django.utils.translation import ugettext_lazy as _
+from .base import BaseACL, BaseACLQuerySet
+from ..utils import contains_ip
+
+
+class ACLManager(models.Manager):
+
+    def valid(self):
+        return self.get_queryset().valid()
+
+
+class LoginACL(BaseACL):
+    class ActionChoices(models.TextChoices):
+        reject = 'reject', _('Reject')
+        allow = 'allow', _('Allow')
+
+    # 条件
+    ip_group = models.JSONField(default=list, verbose_name=_('Login IP'))
+    # 动作
+    action = models.CharField(
+        max_length=64, choices=ActionChoices.choices, default=ActionChoices.reject,
+        verbose_name=_('Action')
+    )
+    # 关联
+    user = models.ForeignKey(
+        'users.User', on_delete=models.CASCADE, related_name='login_acls', verbose_name=_('User')
+    )
+
+    objects = ACLManager.from_queryset(BaseACLQuerySet)()
+
+    class Meta:
+        ordering = ('priority', '-date_updated', 'name')
+
+    def __str__(self):
+        return self.name
+
+    @property
+    def action_reject(self):
+        return self.action == self.ActionChoices.reject
+
+    @property
+    def action_allow(self):
+        return self.action == self.ActionChoices.allow
+
+    @staticmethod
+    def allow_user_to_login(user, ip):
+        acl = user.login_acls.valid().first()
+        if not acl:
+            return True
+        is_contained = contains_ip(ip, acl.ip_group)
+        if acl.action_allow and is_contained:
+            return True
+        if acl.action_reject and not is_contained:
+            return True
+        return False

apps/acls/models/login_asset_acl.py

@@ -0,0 +1,102 @@
+from django.db import models
+from django.db.models import Q
+from django.utils.translation import ugettext_lazy as _
+from orgs.mixins.models import OrgModelMixin, OrgManager
+from .base import BaseACL, BaseACLQuerySet
+from ..utils import contains_ip
+
+
+class ACLManager(OrgManager):
+
+    def valid(self):
+        return self.get_queryset().valid()
+
+
+class LoginAssetACL(BaseACL, OrgModelMixin):
+    class ActionChoices(models.TextChoices):
+        login_confirm = 'login_confirm', _('Login confirm')
+
+    # 条件
+    users = models.JSONField(verbose_name=_('User'))
+    system_users = models.JSONField(verbose_name=_('System User'))
+    assets = models.JSONField(verbose_name=_('Asset'))
+    # 动作
+    action = models.CharField(
+        max_length=64, choices=ActionChoices.choices, default=ActionChoices.login_confirm,
+        verbose_name=_('Action')
+    )
+    # 动作: 附加字段
+    # - login_confirm
+    reviewers = models.ManyToManyField(
+        'users.User', related_name='review_login_asset_acls', blank=True,
+        verbose_name=_("Reviewers")
+    )
+
+    objects = ACLManager.from_queryset(BaseACLQuerySet)()
+
+    class Meta:
+        unique_together = ('name', 'org_id')
+        ordering = ('priority', '-date_updated', 'name')
+
+    def __str__(self):
+        return self.name
+
+    @classmethod
+    def filter(cls, user, asset, system_user, action):
+        queryset = cls.objects.filter(action=action)
+        queryset = cls.filter_user(user, queryset)
+        queryset = cls.filter_asset(asset, queryset)
+        queryset = cls.filter_system_user(system_user, queryset)
+        return queryset
+
+    @classmethod
+    def filter_user(cls, user, queryset):
+        queryset = queryset.filter(
+            Q(users__username_group__contains=user.username) |
+            Q(users__username_group__contains='*')
+        )
+        return queryset
+
+    @classmethod
+    def filter_asset(cls, asset, queryset):
+        queryset = queryset.filter(
+            Q(assets__hostname_group__contains=asset.hostname) |
+            Q(assets__hostname_group__contains='*')
+        )
+        ids = [q.id for q in queryset if contains_ip(asset.ip, q.assets.get('ip_group', []))]
+        queryset = cls.objects.filter(id__in=ids)
+        return queryset
+
+    @classmethod
+    def filter_system_user(cls, system_user, queryset):
+        queryset = queryset.filter(
+            Q(system_users__name_group__contains=system_user.name) |
+            Q(system_users__name_group__contains='*')
+        ).filter(
+            Q(system_users__username_group__contains=system_user.username) |
+            Q(system_users__username_group__contains='*')
+        ).filter(
+            Q(system_users__protocol_group__contains=system_user.protocol) |
+            Q(system_users__protocol_group__contains='*')
+        )
+        return queryset
+
+    @classmethod
+    def create_login_asset_confirm_ticket(cls, user, asset, system_user, assignees, org_id):
+        from tickets.const import TicketTypeChoices
+        from tickets.models import Ticket
+        data = {
+            'title': _('Login asset confirm') + ' ({})'.format(user),
+            'type': TicketTypeChoices.login_asset_confirm,
+            'meta': {
+                'apply_login_user': str(user),
+                'apply_login_asset': str(asset),
+                'apply_login_system_user': str(system_user),
+            },
+            'org_id': org_id,
+        }
+        ticket = Ticket.objects.create(**data)
+        ticket.assignees.set(assignees)
+        ticket.open(applicant=user)
+        return ticket
+

apps/acls/serializers/__init__.py

@@ -0,0 +1,3 @@
+from .login_acl import *
+from .login_asset_acl import *
+from .login_asset_check import *

apps/acls/serializers/login_acl.py

@@ -0,0 +1,59 @@
+from django.utils.translation import ugettext as _
+from rest_framework import serializers
+from common.drf.serializers import BulkModelSerializer
+from orgs.utils import current_org
+from ..models import LoginACL
+from ..utils import is_ip_address, is_ip_network,  is_ip_segment
+
+
+__all__ = ['LoginACLSerializer', ]
+
+
+def ip_group_child_validator(ip_group_child):
+    is_valid = ip_group_child == '*' \
+               or is_ip_address(ip_group_child) \
+               or is_ip_network(ip_group_child) \
+               or is_ip_segment(ip_group_child)
+    if not is_valid:
+        error = _('IP address invalid: `{}`').format(ip_group_child)
+        raise serializers.ValidationError(error)
+
+
+class LoginACLSerializer(BulkModelSerializer):
+    ip_group_help_text = _(
+        'Format for comma-delimited string, with * indicating a match all. '
+        'Such as: '
+        '192.168.10.1, 192.168.1.0/24, 10.1.1.1-10.1.1.20, 2001:db8:2de::e13, 2001:db8:1a:1110::/64 '
+    )
+
+    ip_group = serializers.ListField(
+        default=['*'], label=_('IP'), help_text=ip_group_help_text,
+        child=serializers.CharField(max_length=1024, validators=[ip_group_child_validator])
+    )
+    user_display = serializers.ReadOnlyField(source='user.name', label=_('User'))
+    action_display = serializers.ReadOnlyField(source='get_action_display', label=_('Action'))
+
+    class Meta:
+        model = LoginACL
+        fields_mini = ['id', 'name']
+        fields_small = fields_mini + [
+            'priority', 'ip_group', 'action', 'action_display',
+            'is_active',
+            'date_created', 'date_updated',
+            'comment', 'created_by',
+        ]
+        fields_fk = ['user', 'user_display',]
+        fields = fields_small + fields_fk
+        extra_kwargs = {
+            'priority': {'default': 50},
+            'is_active': {'default': True},
+        }
+
+    @staticmethod
+    def validate_user(user):
+        if user not in current_org.get_members():
+            error = _('The user `{}` is not in the current organization: `{}`').format(
+                user, current_org
+            )
+            raise serializers.ValidationError(error)
+        return user

apps/acls/serializers/login_asset_acl.py

@@ -0,0 +1,105 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+from orgs.mixins.serializers import BulkOrgResourceModelSerializer
+from assets.models import SystemUser
+from acls import models
+from orgs.models import Organization
+
+
+__all__ = ['LoginAssetACLSerializer']
+
+
+common_help_text = _('Format for comma-delimited string, with * indicating a match all. ')
+
+
+class LoginAssetACLUsersSerializer(serializers.Serializer):
+    username_group = serializers.ListField(
+        default=['*'], child=serializers.CharField(max_length=128), label=_('Username'),
+        help_text=common_help_text
+    )
+
+
+class LoginAssetACLAssestsSerializer(serializers.Serializer):
+    ip_group_help_text = _(
+        'Format for comma-delimited string, with * indicating a match all. '
+        'Such as: '
+        '192.168.10.1, 192.168.1.0/24, 10.1.1.1-10.1.1.20, 2001:db8:2de::e13, 2001:db8:1a:1110::/64 '
+        '(Domain name support)'
+    )
+
+    ip_group = serializers.ListField(
+        default=['*'], child=serializers.CharField(max_length=1024), label=_('IP'),
+        help_text=ip_group_help_text
+    )
+    hostname_group = serializers.ListField(
+        default=['*'], child=serializers.CharField(max_length=128), label=_('Hostname'),
+        help_text=common_help_text
+    )
+
+
+class LoginAssetACLSystemUsersSerializer(serializers.Serializer):
+    protocol_group_help_text = _(
+        'Format for comma-delimited string, with * indicating a match all. '
+        'Protocol options: {}'
+    )
+
+    name_group = serializers.ListField(
+        default=['*'], child=serializers.CharField(max_length=128), label=_('Name'),
+        help_text=common_help_text
+    )
+    username_group = serializers.ListField(
+        default=['*'], child=serializers.CharField(max_length=128), label=_('Username'),
+        help_text=common_help_text
+    )
+    protocol_group = serializers.ListField(
+        default=['*'], child=serializers.CharField(max_length=16), label=_('Protocol'),
+        help_text=protocol_group_help_text.format(
+            ', '.join([SystemUser.PROTOCOL_SSH, SystemUser.PROTOCOL_TELNET])
+        )
+    )
+
+    @staticmethod
+    def validate_protocol_group(protocol_group):
+        unsupported_protocols = set(protocol_group) - set(SystemUser.ASSET_CATEGORY_PROTOCOLS + ['*'])
+        if unsupported_protocols:
+            error = _('Unsupported protocols: {}').format(unsupported_protocols)
+            raise serializers.ValidationError(error)
+        return protocol_group
+
+
+class LoginAssetACLSerializer(BulkOrgResourceModelSerializer):
+    users = LoginAssetACLUsersSerializer()
+    assets = LoginAssetACLAssestsSerializer()
+    system_users = LoginAssetACLSystemUsersSerializer()
+    reviewers_amount = serializers.IntegerField(read_only=True, source='reviewers.count')
+    action_display = serializers.ReadOnlyField(source='get_action_display', label=_('Action'))
+
+    class Meta:
+        model = models.LoginAssetACL
+        fields_mini = ['id', 'name']
+        fields_small = fields_mini + [
+            'users', 'system_users', 'assets',
+            'is_active',
+            'date_created', 'date_updated',
+            'priority', 'action', 'action_display', 'comment', 'created_by', 'org_id'
+        ]
+        fields_m2m = ['reviewers', 'reviewers_amount']
+        fields = fields_small + fields_m2m
+        extra_kwargs = {
+            "reviewers": {'allow_null': False, 'required': True},
+            'priority': {'default': 50},
+            'is_active': {'default': True},
+        }
+
+    def validate_reviewers(self, reviewers):
+        org_id = self.fields['org_id'].default()
+        org = Organization.get_instance(org_id)
+        if not org:
+            error = _('The organization `{}` does not exist'.format(org_id))
+            raise serializers.ValidationError(error)
+        users = org.get_members()
+        valid_reviewers = list(set(reviewers) & set(users))
+        if not valid_reviewers:
+            error = _('None of the reviewers belong to Organization `{}`'.format(org.name))
+            raise serializers.ValidationError(error)
+        return valid_reviewers

apps/acls/serializers/login_asset_check.py

@@ -0,0 +1,71 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+from orgs.utils import tmp_to_root_org
+from common.utils import get_object_or_none, lazyproperty
+from users.models import User
+from assets.models import Asset, SystemUser
+
+
+__all__ = ['LoginAssetCheckSerializer']
+
+
+class LoginAssetCheckSerializer(serializers.Serializer):
+    user_id = serializers.UUIDField(required=True, allow_null=False)
+    asset_id = serializers.UUIDField(required=True, allow_null=False)
+    system_user_id = serializers.UUIDField(required=True, allow_null=False)
+    system_user_username = serializers.CharField(max_length=128, default='')
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.user = None
+        self.asset = None
+        self._system_user = None
+        self._system_user_username = None
+
+    def validate_user_id(self, user_id):
+        self.user = self.validate_object_exist(User, user_id)
+        return user_id
+
+    def validate_asset_id(self, asset_id):
+        self.asset = self.validate_object_exist(Asset, asset_id)
+        return asset_id
+
+    def validate_system_user_id(self, system_user_id):
+        self._system_user = self.validate_object_exist(SystemUser, system_user_id)
+        return system_user_id
+
+    def validate_system_user_username(self, system_user_username):
+        system_user_id = self.initial_data.get('system_user_id')
+        system_user = self.validate_object_exist(SystemUser, system_user_id)
+        if self._system_user.login_mode == SystemUser.LOGIN_MANUAL \
+                and not system_user.username \
+                and not system_user.username_same_with_user \
+                and not system_user_username:
+            error = 'Missing parameter: system_user_username'
+            raise serializers.ValidationError(error)
+        self._system_user_username = system_user_username
+        return system_user_username
+
+    @staticmethod
+    def validate_object_exist(model, field_id):
+        with tmp_to_root_org():
+            obj = get_object_or_none(model, pk=field_id)
+        if not obj:
+            error = '{} Model object does not exist'.format(model.__name__)
+            raise serializers.ValidationError(error)
+        return obj
+
+    @lazyproperty
+    def system_user(self):
+        if self._system_user.username_same_with_user:
+            username = self.user.username
+        elif self._system_user.login_mode == SystemUser.LOGIN_MANUAL:
+            username = self._system_user_username
+        else:
+            username = self._system_user.username
+        self._system_user.username = username
+        return self._system_user
+
+    @lazyproperty
+    def org(self):
+        return self.asset.org

apps/acls/urls/__init__.py

@@ -0,0 +1 @@
+from .api_urls import *

apps/acls/urls/api_urls.py

@@ -0,0 +1,18 @@
+from django.urls import path
+from rest_framework_bulk.routes import BulkRouter
+from .. import api
+
+
+app_name = 'acls'
+
+
+router = BulkRouter()
+router.register(r'login-acls', api.LoginACLViewSet, 'login-acl')
+router.register(r'login-asset-acls', api.LoginAssetACLViewSet, 'login-asset-acl')
+
+urlpatterns = [
+    path('login-asset/check/', api.LoginAssetCheckAPI.as_view(), name='login-asset-check'),
+    path('login-asset-confirm/<uuid:pk>/status/', api.LoginAssetConfirmStatusAPI.as_view(), name='login-asset-confirm-status')
+]
+
+urlpatterns += router.urls

apps/acls/utils.py

@@ -0,0 +1,68 @@
+from ipaddress import ip_network, ip_address
+
+
+def is_ip_address(address):
+    """ 192.168.10.1 """
+    try:
+        ip_address(address)
+    except ValueError:
+        return False
+    else:
+        return True
+
+
+def is_ip_network(ip):
+    """ 192.168.1.0/24 """
+    try:
+        ip_network(ip)
+    except ValueError:
+        return False
+    else:
+        return True
+
+
+def is_ip_segment(ip):
+    """ 10.1.1.1-10.1.1.20 """
+    if '-' not in ip:
+        return False
+    ip_address1, ip_address2 = ip.split('-')
+    return is_ip_address(ip_address1) and is_ip_address(ip_address2)
+
+
+def in_ip_segment(ip, ip_segment):
+    ip1, ip2 = ip_segment.split('-')
+    ip1 = int(ip_address(ip1))
+    ip2 = int(ip_address(ip2))
+    ip = int(ip_address(ip))
+    return min(ip1, ip2) <= ip <= max(ip1, ip2)
+
+
+def contains_ip(ip, ip_group):
+    """
+    ip_group:
+    [192.168.10.1, 192.168.1.0/24, 10.1.1.1-10.1.1.20, 2001:db8:2de::e13, 2001:db8:1a:1110::/64.]
+
+    """
+
+    if '*' in ip_group:
+        return True
+
+    for _ip in ip_group:
+        if is_ip_address(_ip):
+            # 192.168.10.1
+            if ip == _ip:
+                return True
+        elif is_ip_network(_ip) and is_ip_address(ip):
+            # 192.168.1.0/24
+            if ip_address(ip) in ip_network(_ip):
+                return True
+        elif is_ip_segment(_ip) and is_ip_address(ip):
+            # 10.1.1.1-10.1.1.20
+            if in_ip_segment(ip, _ip):
+                return True
+        else:
+            # is domain name
+            if ip == _ip:
+                return True
+
+    return False

apps/applications/api/__init__.py

@@ -1,2 +1,4 @@
+from .application import *
+from .application_user import *
+from .mixin import *
 from .remote_app import *
-from .database_app import *

apps/applications/api/application.py

@@ -0,0 +1,19 @@
+# coding: utf-8
+#
+
+from orgs.mixins.api import OrgBulkModelViewSet
+
+from ..hands import IsOrgAdminOrAppUser
+from .. import serializers
+from ..models import Application
+
+
+__all__ = ['ApplicationViewSet']
+
+
+class ApplicationViewSet(OrgBulkModelViewSet):
+    model = Application
+    filterset_fields = ('name', 'type', 'category')
+    search_fields = filterset_fields
+    permission_classes = (IsOrgAdminOrAppUser,)
+    serializer_class = serializers.ApplicationSerializer

apps/applications/api/application_user.py

@@ -0,0 +1,55 @@
+# coding: utf-8
+#
+
+from rest_framework import generics
+from django.conf import settings
+
+from ..hands import IsOrgAdminOrAppUser, IsOrgAdmin, NeedMFAVerify
+from .. import serializers
+from ..models import Application, ApplicationUser
+from perms.models import ApplicationPermission
+
+
+class ApplicationUserListApi(generics.ListAPIView):
+    permission_classes = (IsOrgAdmin, )
+    filterset_fields = ('name', 'username')
+    search_fields = filterset_fields
+    serializer_class = serializers.ApplicationUserSerializer
+    _application = None
+
+    @property
+    def application(self):
+        if self._application is None:
+            app_id = self.request.query_params.get('application_id')
+            if app_id:
+                self._application = Application.objects.get(id=app_id)
+        return self._application
+
+    def get_serializer_context(self):
+        context = super().get_serializer_context()
+        context.update({
+            'application': self.application
+        })
+        return context
+
+    def get_queryset(self):
+        queryset = ApplicationUser.objects.none()
+        if not self.application:
+            return queryset
+        system_user_ids = ApplicationPermission.objects.filter(applications=self.application)\
+            .values_list('system_users', flat=True)
+        if not system_user_ids:
+            return queryset
+        queryset = ApplicationUser.objects.filter(id__in=system_user_ids)
+        return queryset
+
+
+class ApplicationUserAuthInfoListApi(ApplicationUserListApi):
+    serializer_class = serializers.ApplicationUserWithAuthInfoSerializer
+    http_method_names = ['get']
+    permission_classes = [IsOrgAdminOrAppUser]
+
+    def get_permissions(self):
+        if settings.SECURITY_VIEW_AUTH_NEED_MFA:
+            self.permission_classes = [IsOrgAdminOrAppUser, NeedMFAVerify]
+        return super().get_permissions()

apps/applications/api/database_app.py

@@ -1,20 +0,0 @@
-# coding: utf-8
-# 
-
-from orgs.mixins.api import OrgBulkModelViewSet
-
-from .. import models
-from .. import serializers
-from ..hands import IsOrgAdminOrAppUser
-
-__all__ = [
-    'DatabaseAppViewSet',
-]
-
-
-class DatabaseAppViewSet(OrgBulkModelViewSet):
-    model = models.DatabaseApp
-    filter_fields = ('name',)
-    search_fields = filter_fields
-    permission_classes = (IsOrgAdminOrAppUser,)
-    serializer_class = serializers.DatabaseAppSerializer

apps/applications/api/mixin.py

@@ -0,0 +1,89 @@
+from orgs.models import Organization
+
+
+__all__ = ['SerializeApplicationToTreeNodeMixin']
+
+
+class SerializeApplicationToTreeNodeMixin:
+
+    @staticmethod
+    def _serialize_db(db):
+        return {
+            'id': db.id,
+            'name': db.name,
+            'title': db.name,
+            'pId': '',
+            'open': False,
+            'iconSkin': 'database',
+            'meta': {'type': 'database_app'}
+        }
+
+    @staticmethod
+    def _serialize_remote_app(remote_app):
+        return {
+            'id': remote_app.id,
+            'name': remote_app.name,
+            'title': remote_app.name,
+            'pId': '',
+            'open': False,
+            'isParent': False,
+            'iconSkin': 'chrome',
+            'meta': {'type': 'remote_app'}
+        }
+
+    @staticmethod
+    def _serialize_cloud(cloud):
+        return {
+            'id': cloud.id,
+            'name': cloud.name,
+            'title': cloud.name,
+            'pId': '',
+            'open': False,
+            'isParent': False,
+            'iconSkin': 'k8s',
+            'meta': {'type': 'k8s_app'}
+        }
+
+    def _serialize_application(self, application):
+        method_name = f'_serialize_{application.category}'
+        data = getattr(self, method_name)(application)
+        data.update({
+            'pId': application.org.id,
+            'org_name': application.org_name
+        })
+        return data
+
+    def serialize_applications(self, applications):
+        data = [self._serialize_application(application) for application in applications]
+        return data
+
+    @staticmethod
+    def _serialize_organization(org):
+        return {
+            'id': org.id,
+            'name': org.name,
+            'title': org.name,
+            'pId': '',
+            'open': True,
+            'isParent': True,
+            'meta': {
+                'type': 'node'
+            }
+        }
+
+    def serialize_organizations(self, organizations):
+        data = [self._serialize_organization(org) for org in organizations]
+        return data
+
+    @staticmethod
+    def filter_organizations(applications):
+        organization_ids = set(applications.values_list('org_id', flat=True))
+        organizations = [Organization.get_instance(org_id) for org_id in organization_ids]
+        return organizations
+
+    def serialize_applications_with_org(self, applications):
+        organizations = self.filter_organizations(applications)
+        data_organizations = self.serialize_organizations(organizations)
+        data_applications = self.serialize_applications(applications)
+        data = data_organizations + data_applications
+        return data

apps/applications/api/remote_app.py

@@ -1,27 +1,19 @@
 # coding: utf-8
 #
 
-from orgs.mixins.api import OrgBulkModelViewSet
 from orgs.mixins import generics
-from ..hands import IsOrgAdmin, IsAppUser
-from ..models import RemoteApp
-from ..serializers import RemoteAppSerializer, RemoteAppConnectionInfoSerializer
+from ..hands import IsAppUser
+from .. import models
+from ..serializers import RemoteAppConnectionInfoSerializer
+from ..permissions import IsRemoteApp
 
 
 __all__ = [
-    'RemoteAppViewSet', 'RemoteAppConnectionInfoApi',
+    'RemoteAppConnectionInfoApi',
 ]
 
 
-class RemoteAppViewSet(OrgBulkModelViewSet):
-    model = RemoteApp
-    filter_fields = ('name', 'type', 'comment')
-    search_fields = filter_fields
-    permission_classes = (IsOrgAdmin,)
-    serializer_class = RemoteAppSerializer
-
-
 class RemoteAppConnectionInfoApi(generics.RetrieveAPIView):
-    model = RemoteApp
-    permission_classes = (IsAppUser, )
+    model = models.Application
+    permission_classes = (IsAppUser, IsRemoteApp)
     serializer_class = RemoteAppConnectionInfoSerializer

apps/applications/const.py

@@ -1,63 +1,49 @@
 #  coding: utf-8
 #
 
+from django.db.models import TextChoices
 from django.utils.translation import ugettext_lazy as _
 
 
-# RemoteApp
+class ApplicationCategoryChoices(TextChoices):
+    db = 'db', _('Database')
+    remote_app = 'remote_app', _('Remote app')
+    cloud = 'cloud', 'Cloud'
 
-REMOTE_APP_BOOT_PROGRAM_NAME = '||jmservisor'
-
-REMOTE_APP_TYPE_CHROME = 'chrome'
-REMOTE_APP_TYPE_MYSQL_WORKBENCH = 'mysql_workbench'
-REMOTE_APP_TYPE_VMWARE_CLIENT = 'vmware_client'
-REMOTE_APP_TYPE_CUSTOM = 'custom'
-
-# Fields attribute write_only default => False
-
-REMOTE_APP_TYPE_CHROME_FIELDS = [
-    {'name': 'chrome_target'},
-    {'name': 'chrome_username'},
-    {'name': 'chrome_password', 'write_only': True}
-]
-REMOTE_APP_TYPE_MYSQL_WORKBENCH_FIELDS = [
-    {'name': 'mysql_workbench_ip'},
-    {'name': 'mysql_workbench_name'},
-    {'name': 'mysql_workbench_username'},
-    {'name': 'mysql_workbench_password', 'write_only': True}
-]
-REMOTE_APP_TYPE_VMWARE_CLIENT_FIELDS = [
-    {'name': 'vmware_target'},
-    {'name': 'vmware_username'},
-    {'name': 'vmware_password', 'write_only': True}
-]
-REMOTE_APP_TYPE_CUSTOM_FIELDS = [
-    {'name': 'custom_cmdline'},
-    {'name': 'custom_target'},
-    {'name': 'custom_username'},
-    {'name': 'custom_password', 'write_only': True}
-]
-
-REMOTE_APP_TYPE_FIELDS_MAP = {
-    REMOTE_APP_TYPE_CHROME: REMOTE_APP_TYPE_CHROME_FIELDS,
-    REMOTE_APP_TYPE_MYSQL_WORKBENCH: REMOTE_APP_TYPE_MYSQL_WORKBENCH_FIELDS,
-    REMOTE_APP_TYPE_VMWARE_CLIENT: REMOTE_APP_TYPE_VMWARE_CLIENT_FIELDS,
-    REMOTE_APP_TYPE_CUSTOM: REMOTE_APP_TYPE_CUSTOM_FIELDS
-}
-
-REMOTE_APP_TYPE_CHOICES = (
-    (REMOTE_APP_TYPE_CHROME, 'Chrome'),
-    (REMOTE_APP_TYPE_MYSQL_WORKBENCH, 'MySQL Workbench'),
-    (REMOTE_APP_TYPE_VMWARE_CLIENT, 'vSphere Client'),
-    (REMOTE_APP_TYPE_CUSTOM, _('Custom')),
-)
+    @classmethod
+    def get_label(cls, category):
+        return dict(cls.choices).get(category, '')
 
 
-# DatabaseApp
+class ApplicationTypeChoices(TextChoices):
+    # db category
+    mysql = 'mysql', 'MySQL'
+    oracle = 'oracle', 'Oracle'
+    pgsql = 'postgresql', 'PostgreSQL'
+    mariadb = 'mariadb', 'MariaDB'
 
+    # remote-app category
+    chrome = 'chrome', 'Chrome'
+    mysql_workbench = 'mysql_workbench', 'MySQL Workbench'
+    vmware_client = 'vmware_client', 'vSphere Client'
+    custom = 'custom', _('Custom')
 
-DATABASE_APP_TYPE_MYSQL = 'mysql'
+    # cloud category
+    k8s = 'k8s', 'Kubernetes'
+
+    @classmethod
+    def get_label(cls, tp):
+        return dict(cls.choices).get(tp, '')
+
+    @classmethod
+    def db_types(cls):
+        return [cls.mysql.value, cls.oracle.value, cls.pgsql.value, cls.mariadb.value]
+
+    @classmethod
+    def remote_app_types(cls):
+        return [cls.chrome.value, cls.mysql_workbench.value, cls.vmware_client.value, cls.custom.value]
+
+    @classmethod
+    def cloud_types(cls):
+        return [cls.k8s.value]
 
-DATABASE_APP_TYPE_CHOICES = (
-    (DATABASE_APP_TYPE_MYSQL, 'MySQL'),
-)

apps/applications/hands.py

@@ -11,5 +11,5 @@
 """
 
 
-from common.permissions import IsAppUser, IsOrgAdmin, IsValidUser, IsOrgAdminOrAppUser
+from common.permissions import IsAppUser, IsOrgAdmin, IsValidUser, IsOrgAdminOrAppUser, NeedMFAVerify
 from users.models import User, UserGroup

apps/applications/migrations/0005_k8sapp.py

@@ -0,0 +1,34 @@
+# Generated by Django 2.2.13 on 2020-08-07 07:13
+
+from django.db import migrations, models
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('applications', '0004_auto_20191218_1705'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='K8sApp',
+            fields=[
+                ('created_by', models.CharField(blank=True, max_length=32, null=True, verbose_name='Created by')),
+                ('updated_by', models.CharField(blank=True, max_length=32, null=True, verbose_name='Updated by')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+                ('date_updated', models.DateTimeField(auto_now=True, verbose_name='Date updated')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
+                ('name', models.CharField(max_length=128, verbose_name='Name')),
+                ('type', models.CharField(choices=[('k8s', 'Kubernetes')], default='k8s', max_length=128, verbose_name='Type')),
+                ('cluster', models.CharField(max_length=1024, verbose_name='Cluster')),
+                ('comment', models.TextField(blank=True, default='', max_length=128, verbose_name='Comment')),
+            ],
+            options={
+                'verbose_name': 'KubernetesApp',
+                'ordering': ('name',),
+                'unique_together': {('org_id', 'name')},
+            },
+        ),
+    ]

apps/applications/migrations/0006_application.py

@@ -0,0 +1,140 @@
+# Generated by Django 2.2.13 on 2020-10-19 12:01
+
+from django.db import migrations, models
+import django.db.models.deletion
+import django_mysql.models
+import uuid
+
+
+CATEGORY_DB_LIST = ['mysql', 'oracle', 'postgresql', 'mariadb']
+CATEGORY_REMOTE_LIST = ['chrome', 'mysql_workbench', 'vmware_client', 'custom']
+CATEGORY_CLOUD_LIST = ['k8s']
+
+CATEGORY_DB = 'db'
+CATEGORY_REMOTE = 'remote_app'
+CATEGORY_CLOUD = 'cloud'
+CATEGORY_LIST = [CATEGORY_DB, CATEGORY_REMOTE, CATEGORY_CLOUD]
+
+
+def get_application_category(old_app):
+    _type = old_app.type
+    if _type in CATEGORY_DB_LIST:
+        category = CATEGORY_DB
+    elif _type in CATEGORY_REMOTE_LIST:
+        category = CATEGORY_REMOTE
+    elif _type in CATEGORY_CLOUD_LIST:
+        category = CATEGORY_CLOUD
+    else:
+        category = None
+    return category
+
+
+def common_to_application_json(old_app):
+    category = get_application_category(old_app)
+    date_updated = old_app.date_updated if hasattr(old_app, 'date_updated') else old_app.date_created
+    return {
+        'id': old_app.id,
+        'name': old_app.name,
+        'type': old_app.type,
+        'category': category,
+        'comment': old_app.comment,
+        'created_by': old_app.created_by,
+        'date_created': old_app.date_created,
+        'date_updated': date_updated,
+        'org_id': old_app.org_id
+    }
+
+
+def db_to_application_json(database):
+    app_json = common_to_application_json(database)
+    app_json.update({
+        'attrs': {
+            'host': database.host,
+            'port': database.port,
+            'database': database.database
+        }
+    })
+    return app_json
+
+
+def remote_to_application_json(remote):
+    app_json = common_to_application_json(remote)
+    attrs = {
+        'asset': str(remote.asset.id),
+        'path': remote.path,
+    }
+    attrs.update(remote.params)
+    app_json.update({
+        'attrs': attrs
+    })
+    return app_json
+
+
+def k8s_to_application_json(k8s):
+    app_json = common_to_application_json(k8s)
+    app_json.update({
+        'attrs': {
+            'cluster': k8s.cluster
+        }
+    })
+    return app_json
+
+
+def migrate_and_integrate_applications(apps, schema_editor):
+    db_alias = schema_editor.connection.alias
+
+    database_app_model = apps.get_model("applications", "DatabaseApp")
+    remote_app_model = apps.get_model("applications", "RemoteApp")
+    k8s_app_model = apps.get_model("applications", "K8sApp")
+
+    database_apps = database_app_model.objects.using(db_alias).all()
+    remote_apps = remote_app_model.objects.using(db_alias).all()
+    k8s_apps = k8s_app_model.objects.using(db_alias).all()
+
+    database_applications = [db_to_application_json(db_app) for db_app in database_apps]
+    remote_applications = [remote_to_application_json(remote_app) for remote_app in remote_apps]
+    k8s_applications = [k8s_to_application_json(k8s_app) for k8s_app in k8s_apps]
+
+    applications_json = database_applications + remote_applications + k8s_applications
+    application_model = apps.get_model("applications", "Application")
+    applications = [
+        application_model(**application_json)
+        for application_json in applications_json
+        if application_json['category'] in CATEGORY_LIST
+    ]
+    for application in applications:
+        if application_model.objects.using(db_alias).filter(name=application.name).exists():
+            application.name = '{}-{}'.format(application.name, application.type)
+        application.save()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0057_fill_node_value_assets_amount_and_parent_key'),
+        ('applications', '0005_k8sapp'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Application',
+            fields=[
+                ('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('created_by', models.CharField(blank=True, max_length=32, null=True, verbose_name='Created by')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+                ('date_updated', models.DateTimeField(auto_now=True, verbose_name='Date updated')),
+                ('name', models.CharField(max_length=128, verbose_name='Name')),
+                ('category', models.CharField(choices=[('db', 'Database'), ('remote_app', 'Remote app'), ('cloud', 'Cloud')], max_length=16, verbose_name='Category')),
+                ('type', models.CharField(choices=[('mysql', 'MySQL'), ('oracle', 'Oracle'), ('postgresql', 'PostgreSQL'), ('mariadb', 'MariaDB'), ('chrome', 'Chrome'), ('mysql_workbench', 'MySQL Workbench'), ('vmware_client', 'vSphere Client'), ('custom', 'Custom'), ('k8s', 'Kubernetes')], max_length=16, verbose_name='Type')),
+                ('attrs', django_mysql.models.JSONField(default=dict)),
+                ('comment', models.TextField(blank=True, default='', max_length=128, verbose_name='Comment')),
+                ('domain', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='applications', to='assets.Domain', verbose_name='Domain')),
+            ],
+            options={
+                'ordering': ('name',),
+                'unique_together': {('org_id', 'name')},
+            },
+        ),
+        migrations.RunPython(migrate_and_integrate_applications),
+    ]

apps/applications/migrations/0007_auto_20201119_1110.py

@@ -0,0 +1,18 @@
+# Generated by Django 3.1 on 2020-11-19 03:10
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('applications', '0006_application'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='application',
+            name='attrs',
+            field=models.JSONField(),
+        ),
+    ]

apps/applications/migrations/0008_auto_20210104_0435.py

@@ -0,0 +1,28 @@
+# Generated by Django 3.1 on 2021-01-03 20:35
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('perms', '0017_auto_20210104_0435'),
+        ('applications', '0007_auto_20201119_1110'),
+    ]
+
+    operations = [
+        migrations.DeleteModel(
+            name='DatabaseApp',
+        ),
+        migrations.DeleteModel(
+            name='K8sApp',
+        ),
+        migrations.AlterField(
+            model_name='application',
+            name='attrs',
+            field=models.JSONField(default=dict, verbose_name='Attrs'),
+        ),
+        migrations.DeleteModel(
+            name='RemoteApp',
+        ),
+    ]

apps/applications/models/__init__.py

@@ -1,2 +1 @@
-from .remote_app import *
-from .database_app import *
+from .application import *

apps/applications/models/application.py

@@ -0,0 +1,75 @@
+from django.db import models
+from django.utils.translation import ugettext_lazy as _
+
+from orgs.mixins.models import OrgModelMixin
+from common.mixins import CommonModelMixin
+from assets.models import Asset, SystemUser
+from .. import const
+
+
+class Application(CommonModelMixin, OrgModelMixin):
+    name = models.CharField(max_length=128, verbose_name=_('Name'))
+    category = models.CharField(
+        max_length=16, choices=const.ApplicationCategoryChoices.choices, verbose_name=_('Category')
+    )
+    type = models.CharField(
+        max_length=16, choices=const.ApplicationTypeChoices.choices, verbose_name=_('Type')
+    )
+    domain = models.ForeignKey(
+        'assets.Domain', null=True, blank=True, related_name='applications',
+        on_delete=models.SET_NULL, verbose_name=_("Domain"),
+    )
+    attrs = models.JSONField(default=dict, verbose_name=_('Attrs'))
+    comment = models.TextField(
+        max_length=128, default='', blank=True, verbose_name=_('Comment')
+    )
+
+    class Meta:
+        unique_together = [('org_id', 'name')]
+        ordering = ('name',)
+
+    def __str__(self):
+        category_display = self.get_category_display()
+        type_display = self.get_type_display()
+        return f'{self.name}({type_display})[{category_display}]'
+
+    @property
+    def category_remote_app(self):
+        return self.category == const.ApplicationCategoryChoices.remote_app.value
+
+    def get_rdp_remote_app_setting(self):
+        from applications.serializers.attrs import get_serializer_class_by_application_type
+        if not self.category_remote_app:
+            raise ValueError(f"Not a remote app application: {self.name}")
+        serializer_class = get_serializer_class_by_application_type(self.type)
+        fields = serializer_class().get_fields()
+
+        parameters = [self.type]
+        for field_name in list(fields.keys()):
+            if field_name in ['asset']:
+                continue
+            value = self.attrs.get(field_name)
+            if not value:
+                continue
+            if field_name == 'path':
+                value = '\"%s\"' % value
+            parameters.append(str(value))
+
+        parameters = ' '.join(parameters)
+        return {
+            'program': '||jmservisor',
+            'working_directory': '',
+            'parameters': parameters
+        }
+
+    def get_remote_app_asset(self):
+        asset_id = self.attrs.get('asset')
+        if not asset_id:
+            raise ValueError("Remote App not has asset attr")
+        asset = Asset.objects.filter(id=asset_id).first()
+        return asset
+
+
+class ApplicationUser(SystemUser):
+    class Meta:
+        proxy = True

apps/applications/models/database_app.py

@@ -1,42 +0,0 @@
-# coding: utf-8
-#
-
-import uuid
-from django.db import models
-from django.utils.translation import ugettext_lazy as _
-
-from orgs.mixins.models import OrgModelMixin
-from common.mixins import CommonModelMixin
-from .. import const
-
-
-__all__ = ['DatabaseApp']
-
-
-class DatabaseApp(CommonModelMixin, OrgModelMixin):
-    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
-    name = models.CharField(max_length=128, verbose_name=_('Name'))
-    type = models.CharField(
-        default=const.DATABASE_APP_TYPE_MYSQL,
-        choices=const.DATABASE_APP_TYPE_CHOICES,
-        max_length=128, verbose_name=_('Type')
-    )
-    host = models.CharField(
-        max_length=128, verbose_name=_('Host'), db_index=True
-    )
-    port = models.IntegerField(default=3306, verbose_name=_('Port'))
-    database = models.CharField(
-        max_length=128, blank=True, null=True, verbose_name=_('Database'),
-        db_index=True
-    )
-    comment = models.TextField(
-        max_length=128, default='', blank=True, verbose_name=_('Comment')
-    )
-
-    def __str__(self):
-        return self.name
-
-    class Meta:
-        unique_together = [('org_id', 'name'), ]
-        verbose_name = _("DatabaseApp")
-        ordering = ('name', )

apps/applications/models/remote_app.py

@@ -1,78 +0,0 @@
-# coding: utf-8
-#
-
-import uuid
-from django.db import models
-from django.utils.translation import ugettext_lazy as _
-
-from orgs.mixins.models import OrgModelMixin
-from common.fields.model import EncryptJsonDictTextField
-
-from .. import const
-
-
-__all__ = [
-    'RemoteApp',
-]
-
-
-class RemoteApp(OrgModelMixin):
-    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
-    name = models.CharField(max_length=128, verbose_name=_('Name'))
-    asset = models.ForeignKey(
-        'assets.Asset', on_delete=models.CASCADE, verbose_name=_('Asset')
-    )
-    type = models.CharField(
-        default=const.REMOTE_APP_TYPE_CHROME,
-        choices=const.REMOTE_APP_TYPE_CHOICES,
-        max_length=128, verbose_name=_('App type')
-    )
-    path = models.CharField(
-        max_length=128, blank=False, null=False,
-        verbose_name=_('App path')
-    )
-    params = EncryptJsonDictTextField(
-        max_length=4096, default={}, blank=True, null=True,
-        verbose_name=_('Parameters')
-    )
-    created_by = models.CharField(
-        max_length=32, null=True, blank=True, verbose_name=_('Created by')
-    )
-    date_created = models.DateTimeField(
-        auto_now_add=True, null=True, blank=True, verbose_name=_('Date created')
-    )
-    comment = models.TextField(
-        max_length=128, default='', blank=True, verbose_name=_('Comment')
-    )
-
-    class Meta:
-        verbose_name = _("RemoteApp")
-        unique_together = [('org_id', 'name')]
-        ordering = ('name', )
-
-    def __str__(self):
-        return self.name
-
-    @property
-    def parameters(self):
-        """
-        返回Guacamole需要的RemoteApp配置参数信息中的parameters参数
-        """
-        _parameters = list()
-        _parameters.append(self.type)
-        path = '\"%s\"' % self.path
-        _parameters.append(path)
-        for field in const.REMOTE_APP_TYPE_FIELDS_MAP[self.type]:
-            value = self.params.get(field['name'])
-            if value is None:
-                continue
-            _parameters.append(value)
-        _parameters = ' '.join(_parameters)
-        return _parameters
-
-    @property
-    def asset_info(self):
-        return {
-            'id': self.asset.id,
-            'hostname': self.asset.hostname
-        }

apps/applications/permissions.py

@@ -0,0 +1,9 @@
+from rest_framework import permissions
+
+
+__all__ = ['IsRemoteApp']
+
+
+class IsRemoteApp(permissions.BasePermission):
+    def has_object_permission(self, request, view, obj):
+        return obj.category_remote_app

apps/applications/serializers/__init__.py

@@ -1,2 +1,2 @@
+from .application import *
 from .remote_app import *
-from .database_app import *

apps/applications/serializers/application.py

@@ -0,0 +1,108 @@
+# coding: utf-8
+#
+
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+from orgs.mixins.serializers import BulkOrgResourceModelSerializer
+from common.drf.serializers import MethodSerializer
+from .attrs import category_serializer_classes_mapping, type_serializer_classes_mapping
+from assets.serializers import SystemUserSerializer
+from .. import models
+
+__all__ = [
+    'ApplicationSerializer', 'ApplicationSerializerMixin',
+    'ApplicationUserSerializer', 'ApplicationUserWithAuthInfoSerializer'
+]
+
+
+class ApplicationSerializerMixin(serializers.Serializer):
+    attrs = MethodSerializer()
+
+    def get_attrs_serializer(self):
+        default_serializer = serializers.Serializer(read_only=True)
+        if isinstance(self.instance, models.Application):
+            _type = self.instance.type
+            _category = self.instance.category
+        else:
+            _type = self.context['request'].query_params.get('type')
+            _category = self.context['request'].query_params.get('category')
+
+        if _type:
+            serializer_class = type_serializer_classes_mapping.get(_type)
+        elif _category:
+            serializer_class = category_serializer_classes_mapping.get(_category)
+        else:
+            serializer_class = default_serializer
+
+        if not serializer_class:
+            serializer_class = default_serializer
+
+        if isinstance(serializer_class, type):
+            serializer = serializer_class()
+        else:
+            serializer = serializer_class
+        return serializer
+
+
+class ApplicationSerializer(ApplicationSerializerMixin, BulkOrgResourceModelSerializer):
+    category_display = serializers.ReadOnlyField(source='get_category_display', label=_('Category(Display)'))
+    type_display = serializers.ReadOnlyField(source='get_type_display', label=_('Type(Dispaly)'))
+
+    class Meta:
+        model = models.Application
+        fields_mini = ['id', 'name']
+        fields_small = fields_mini + [
+            'category', 'category_display', 'type', 'type_display', 'attrs',
+            'date_created', 'date_updated',
+            'created_by', 'comment'
+        ]
+        fields_fk = ['domain']
+        fields = fields_small + fields_fk
+        read_only_fields = [
+            'created_by', 'date_created', 'date_updated', 'get_type_display',
+        ]
+
+    def validate_attrs(self, attrs):
+        _attrs = self.instance.attrs if self.instance else {}
+        _attrs.update(attrs)
+        return _attrs
+
+
+class ApplicationUserSerializer(SystemUserSerializer):
+    application_name = serializers.SerializerMethodField(label=_('Application name'))
+    application_category = serializers.SerializerMethodField(label=_('Application category'))
+    application_type = serializers.SerializerMethodField(label=_('Application type'))
+
+    class Meta(SystemUserSerializer.Meta):
+        model = models.ApplicationUser
+        fields_mini = [
+            'id', 'application_name', 'application_category', 'application_type', 'name', 'username'
+        ]
+        fields_small = fields_mini + [
+            'protocol', 'login_mode', 'login_mode_display', 'priority',
+            "username_same_with_user", 'comment',
+        ]
+        fields = fields_small
+        extra_kwargs = {
+            'login_mode_display': {'label': _('Login mode display')},
+            'created_by': {'read_only': True},
+        }
+
+    @property
+    def application(self):
+        return self.context['application']
+
+    def get_application_name(self, obj):
+        return self.application.name
+
+    def get_application_category(self, obj):
+        return self.application.get_category_display()
+
+    def get_application_type(self, obj):
+        return self.application.get_type_display()
+
+
+class ApplicationUserWithAuthInfoSerializer(ApplicationUserSerializer):
+
+    class Meta(ApplicationUserSerializer.Meta):
+        fields = ApplicationUserSerializer.Meta.fields + ['password', 'token']

apps/applications/serializers/attrs/__init__.py

@@ -0,0 +1 @@
+from .attrs import *

apps/applications/serializers/attrs/application_category/__init__.py

@@ -0,0 +1,3 @@
+from .remote_app import *
+from .db import *
+from .cloud import *

apps/applications/serializers/attrs/application_category/cloud.py

@@ -0,0 +1,9 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+
+__all__ = ['CloudSerializer']
+
+
+class CloudSerializer(serializers.Serializer):
+    cluster = serializers.CharField(max_length=1024, label=_('Cluster'), allow_null=True)

apps/applications/serializers/attrs/application_category/db.py

@@ -0,0 +1,15 @@
+# coding: utf-8
+#
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+
+__all__ = ['DBSerializer']
+
+
+class DBSerializer(serializers.Serializer):
+    host = serializers.CharField(max_length=128, label=_('Host'), allow_null=True)
+    port = serializers.IntegerField(label=_('Port'), allow_null=True)
+    database = serializers.CharField(
+        max_length=128, required=True, allow_null=True, label=_('Database')
+    )

apps/applications/serializers/attrs/application_category/remote_app.py

@@ -0,0 +1,52 @@
+# coding: utf-8
+#
+
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+from django.core.exceptions import ObjectDoesNotExist
+
+from common.utils import get_logger, is_uuid
+from assets.models import Asset
+
+logger = get_logger(__file__)
+
+
+__all__ = ['RemoteAppSerializer']
+
+
+class CharPrimaryKeyRelatedField(serializers.PrimaryKeyRelatedField):
+
+    def to_internal_value(self, data):
+        instance = super().to_internal_value(data)
+        return str(instance.id)
+
+    def to_representation(self, value):
+        # value is instance.id
+        if self.pk_field is not None:
+            return self.pk_field.to_representation(value)
+        return value
+
+
+class RemoteAppSerializer(serializers.Serializer):
+    asset_info = serializers.SerializerMethodField()
+    asset = CharPrimaryKeyRelatedField(
+        queryset=Asset.objects, required=False, label=_("Asset"), allow_null=True
+    )
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), allow_null=True
+    )
+
+    @staticmethod
+    def get_asset_info(obj):
+        asset_id = obj.get('asset')
+        if not asset_id or not is_uuid(asset_id):
+            return {}
+        try:
+            asset = Asset.objects.get(id=str(asset_id))
+        except ObjectDoesNotExist as e:
+            logger.error(e)
+            return {}
+        if not asset:
+            return {}
+        asset_info = {'id': str(asset.id), 'hostname': asset.hostname}
+        return asset_info

apps/applications/serializers/attrs/application_type/__init__.py

@@ -0,0 +1,12 @@
+
+from .mysql import *
+from .mariadb import *
+from .oracle import *
+from .pgsql import *
+
+from .chrome import *
+from .mysql_workbench import *
+from .vmware_client import *
+from .custom import *
+
+from .k8s import *

apps/applications/serializers/attrs/application_type/chrome.py

@@ -0,0 +1,26 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['ChromeSerializer']
+
+
+class ChromeSerializer(RemoteAppSerializer):
+    CHROME_PATH = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'
+
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), default=CHROME_PATH, allow_null=True,
+    )
+    chrome_target = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Target URL'), allow_null=True,
+    )
+    chrome_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'), allow_null=True,
+    )
+    chrome_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True
+    )
+

apps/applications/serializers/attrs/application_type/custom.py

@@ -0,0 +1,27 @@
+
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['CustomSerializer']
+
+
+class CustomSerializer(RemoteAppSerializer):
+    custom_cmdline = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Operating parameter'),
+        allow_null=True,
+    )
+    custom_target = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Target url'),
+        allow_null=True,
+    )
+    custom_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'),
+        allow_null=True,
+    )
+    custom_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True,
+    )

apps/applications/serializers/attrs/application_type/k8s.py

@@ -0,0 +1,8 @@
+from ..application_category import CloudSerializer
+
+
+__all__ = ['K8SSerializer']
+
+
+class K8SSerializer(CloudSerializer):
+    pass

apps/applications/serializers/attrs/application_type/mariadb.py

@@ -0,0 +1,8 @@
+from .mysql import MySQLSerializer
+
+
+__all__ = ['MariaDBSerializer']
+
+
+class MariaDBSerializer(MySQLSerializer):
+    pass

apps/applications/serializers/attrs/application_type/mysql.py

@@ -0,0 +1,15 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+from ..application_category import DBSerializer
+
+
+__all__ = ['MySQLSerializer']
+
+
+class MySQLSerializer(DBSerializer):
+    port = serializers.IntegerField(default=3306, label=_('Port'), allow_null=True)
+
+
+
+

apps/applications/serializers/attrs/application_type/mysql_workbench.py

@@ -0,0 +1,36 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['MySQLWorkbenchSerializer']
+
+
+class MySQLWorkbenchSerializer(RemoteAppSerializer):
+    MYSQL_WORKBENCH_PATH = 'C:\Program Files\MySQL\MySQL Workbench 8.0 CE\MySQLWorkbench.exe'
+
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), default=MYSQL_WORKBENCH_PATH,
+        allow_null=True,
+    )
+    mysql_workbench_ip = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('IP'),
+        allow_null=True,
+    )
+    mysql_workbench_port = serializers.IntegerField(
+        required=False, label=_('Port'),
+        allow_null=True,
+    )
+    mysql_workbench_name = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Database'),
+        allow_null=True,
+    )
+    mysql_workbench_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'),
+        allow_null=True,
+    )
+    mysql_workbench_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True,
+    )

apps/applications/serializers/attrs/application_type/oracle.py

@@ -0,0 +1,12 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+from ..application_category import DBSerializer
+
+
+__all__ = ['OracleSerializer']
+
+
+class OracleSerializer(DBSerializer):
+    port = serializers.IntegerField(default=1521, label=_('Port'), allow_null=True)
+

apps/applications/serializers/attrs/application_type/pgsql.py

@@ -0,0 +1,12 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+from ..application_category import DBSerializer
+
+
+__all__ = ['PostgreSerializer']
+
+
+class PostgreSerializer(DBSerializer):
+    port = serializers.IntegerField(default=5432, label=_('Port'), allow_null=True)
+

apps/applications/serializers/attrs/application_type/vmware_client.py

@@ -0,0 +1,32 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['VMwareClientSerializer']
+
+
+class VMwareClientSerializer(RemoteAppSerializer):
+    PATH = r'''
+    C:\Program Files (x86)\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\VpxClient
+    .exe
+    '''
+    VMWARE_CLIENT_PATH = ''.join(PATH.split())
+
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), default=VMWARE_CLIENT_PATH,
+        allow_null=True
+    )
+    vmware_target = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Target URL'),
+        allow_null=True
+    )
+    vmware_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'),
+        allow_null=True
+    )
+    vmware_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True
+    )

apps/applications/serializers/attrs/attrs.py

@@ -0,0 +1,42 @@
+from rest_framework import serializers
+from applications import const
+from . import application_category, application_type
+
+
+__all__ = [
+    'category_serializer_classes_mapping',
+    'type_serializer_classes_mapping',
+    'get_serializer_class_by_application_type',
+]
+
+
+# define `attrs` field `category serializers mapping`
+# ---------------------------------------------------
+
+category_serializer_classes_mapping = {
+    const.ApplicationCategoryChoices.db.value: application_category.DBSerializer,
+    const.ApplicationCategoryChoices.remote_app.value: application_category.RemoteAppSerializer,
+    const.ApplicationCategoryChoices.cloud.value: application_category.CloudSerializer,
+}
+
+# define `attrs` field `type serializers mapping`
+# -----------------------------------------------
+
+type_serializer_classes_mapping = {
+    # db
+    const.ApplicationTypeChoices.mysql.value: application_type.MySQLSerializer,
+    const.ApplicationTypeChoices.mariadb.value: application_type.MariaDBSerializer,
+    const.ApplicationTypeChoices.oracle.value: application_type.OracleSerializer,
+    const.ApplicationTypeChoices.pgsql.value: application_type.PostgreSerializer,
+    # remote-app
+    const.ApplicationTypeChoices.chrome.value: application_type.ChromeSerializer,
+    const.ApplicationTypeChoices.mysql_workbench.value: application_type.MySQLWorkbenchSerializer,
+    const.ApplicationTypeChoices.vmware_client.value: application_type.VMwareClientSerializer,
+    const.ApplicationTypeChoices.custom.value: application_type.CustomSerializer,
+    # cloud
+    const.ApplicationTypeChoices.k8s.value: application_type.K8SSerializer
+}
+
+
+def get_serializer_class_by_application_type(_application_type):
+    return type_serializer_classes_mapping.get(_application_type)

apps/applications/serializers/database_app.py

@@ -1,26 +0,0 @@
-# coding: utf-8
-#
-
-from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-from common.serializers import AdaptedBulkListSerializer
-
-from .. import models
-
-__all__ = [
-    'DatabaseAppSerializer',
-]
-
-
-class DatabaseAppSerializer(BulkOrgResourceModelSerializer):
-
-    class Meta:
-        model = models.DatabaseApp
-        list_serializer_class = AdaptedBulkListSerializer
-        fields = [
-            'id', 'name', 'type', 'get_type_display', 'host', 'port',
-            'database', 'comment', 'created_by', 'date_created', 'date_updated',
-        ]
-        read_only_fields = [
-            'created_by', 'date_created', 'date_updated'
-            'get_type_display',
-        ]

apps/applications/serializers/remote_app.py

@@ -1,86 +1,31 @@
 # coding: utf-8
 #
-
-import copy
 from rest_framework import serializers
-
-from common.serializers import AdaptedBulkListSerializer
-from common.fields.serializer import CustomMetaDictField
-from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-
-from .. import const
-from ..models import RemoteApp
+from common.utils import get_logger
+from ..models import Application
 
 
-__all__ = [
-    'RemoteAppSerializer', 'RemoteAppConnectionInfoSerializer',
-]
+logger = get_logger(__file__)
 
 
-class RemoteAppParamsDictField(CustomMetaDictField):
-    type_fields_map = const.REMOTE_APP_TYPE_FIELDS_MAP
-    default_type = const.REMOTE_APP_TYPE_CHROME
-    convert_key_remove_type_prefix = False
-    convert_key_to_upper = False
-
-
-class RemoteAppSerializer(BulkOrgResourceModelSerializer):
-    params = RemoteAppParamsDictField()
-    type_fields_map = const.REMOTE_APP_TYPE_FIELDS_MAP
-
-    class Meta:
-        model = RemoteApp
-        list_serializer_class = AdaptedBulkListSerializer
-        fields = [
-            'id', 'name', 'asset', 'asset_info', 'type', 'get_type_display',
-            'path', 'params', 'date_created', 'created_by', 'comment',
-        ]
-        read_only_fields = [
-            'created_by', 'date_created', 'asset_info',
-            'get_type_display'
-        ]
-
-    def process_params(self, instance, validated_data):
-        new_params = copy.deepcopy(validated_data.get('params', {}))
-        tp = validated_data.get('type', '')
-
-        if tp != instance.type:
-            return new_params
-
-        old_params = instance.params
-        fields = self.type_fields_map.get(instance.type, [])
-        for field in fields:
-            if not field.get('write_only', False):
-                continue
-            field_name = field['name']
-            new_value = new_params.get(field_name, '')
-            old_value = old_params.get(field_name, '')
-            field_value = new_value if new_value else old_value
-            new_params[field_name] = field_value
-
-        return new_params
-
-    def update(self, instance, validated_data):
-        params = self.process_params(instance, validated_data)
-        validated_data['params'] = params
-        return super().update(instance, validated_data)
+__all__ = ['RemoteAppConnectionInfoSerializer']
 
 
 class RemoteAppConnectionInfoSerializer(serializers.ModelSerializer):
     parameter_remote_app = serializers.SerializerMethodField()
+    asset = serializers.SerializerMethodField()
 
     class Meta:
-        model = RemoteApp
+        model = Application
         fields = [
             'id', 'name', 'asset', 'parameter_remote_app',
         ]
         read_only_fields = ['parameter_remote_app']
 
+    @staticmethod
+    def get_asset(obj):
+        return obj.attrs.get('asset')
+
     @staticmethod
     def get_parameter_remote_app(obj):
-        parameter = {
-            'program': const.REMOTE_APP_BOOT_PROGRAM_NAME,
-            'working_directory': '',
-            'parameters': obj.parameters,
-        }
-        return parameter
+        return obj.get_rdp_remote_app_setting()

apps/applications/urls/api_urls.py

@@ -1,24 +1,22 @@
 # coding:utf-8
 #
-
-from django.urls import path, re_path
+from django.urls import path
 from rest_framework_bulk.routes import BulkRouter
-
-from common import api as capi
 from .. import api
 
+
 app_name = 'applications'
 
+
 router = BulkRouter()
-router.register(r'remote-apps', api.RemoteAppViewSet, 'remote-app')
-router.register(r'database-apps', api.DatabaseAppViewSet, 'database-app')
+router.register(r'applications', api.ApplicationViewSet, 'application')
+
 
 urlpatterns = [
     path('remote-apps/<uuid:pk>/connection-info/', api.RemoteAppConnectionInfoApi.as_view(), name='remote-app-connection-info'),
+    path('application-users/', api.ApplicationUserListApi.as_view(), name='application-user'),
+    path('application-user-auth-infos/', api.ApplicationUserAuthInfoListApi.as_view(), name='application-user-auth-info')
 ]
 
-old_version_urlpatterns = [
-    re_path('(?P<resource>remote-app)/.*', capi.redirect_plural_name_api)
-]
 
-urlpatterns += router.urls + old_version_urlpatterns
+urlpatterns += router.urls

apps/applications/urls/views_urls.py

@@ -1,7 +0,0 @@
-# coding:utf-8
-from django.urls import path
-
-app_name = 'applications'
-
-urlpatterns = [
-]

apps/assets/api/__init__.py

@@ -1,3 +1,4 @@
+from .mixin import *
 from .admin_user import *
 from .asset import *
 from .label import *

apps/assets/api/admin_user.py

@@ -29,10 +29,14 @@ class AdminUserViewSet(OrgBulkModelViewSet):
     Admin user api set, for add,delete,update,list,retrieve resource
     """
     model = AdminUser
-    filter_fields = ("name", "username")
-    search_fields = filter_fields
+    filterset_fields = ("name", "username")
+    search_fields = filterset_fields
     serializer_class = serializers.AdminUserSerializer
     permission_classes = (IsOrgAdmin,)
+    serializer_classes = {
+        'default': serializers.AdminUserSerializer,
+        'retrieve': serializers.AdminUserDetailSerializer,
+    }
 
     def get_queryset(self):
         queryset = super().get_queryset()
@@ -93,9 +97,8 @@ class AdminUserTestConnectiveApi(generics.RetrieveAPIView):
 class AdminUserAssetsListView(generics.ListAPIView):
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.AssetSimpleSerializer
-    filter_fields = ("hostname", "ip")
-    http_method_names = ['get']
-    search_fields = filter_fields
+    filterset_fields = ("hostname", "ip")
+    search_fields = filterset_fields
 
     def get_object(self):
         pk = self.kwargs.get('pk')

apps/assets/api/asset.py

@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-
+from assets.api import FilterAssetByNodeMixin
 from rest_framework.viewsets import ModelViewSet
 from rest_framework.generics import RetrieveAPIView
 from django.shortcuts import get_object_or_404
@@ -12,28 +12,33 @@ from orgs.mixins import generics
 from ..models import Asset, Node, Platform
 from .. import serializers
 from ..tasks import (
-    update_asset_hardware_info_manual, test_asset_connectivity_manual
+    update_assets_hardware_info_manual, test_assets_connectivity_manual
 )
-from ..filters import AssetByNodeFilterBackend, LabelFilterBackend, IpInFilterBackend
+from ..filters import FilterAssetByNodeFilterBackend, LabelFilterBackend, IpInFilterBackend
 
 
 logger = get_logger(__file__)
 __all__ = [
     'AssetViewSet', 'AssetPlatformRetrieveApi',
     'AssetGatewayListApi', 'AssetPlatformViewSet',
-    'AssetTaskCreateApi',
+    'AssetTaskCreateApi', 'AssetsTaskCreateApi',
 ]
 
 
-class AssetViewSet(OrgBulkModelViewSet):
+class AssetViewSet(FilterAssetByNodeMixin, OrgBulkModelViewSet):
     """
     API endpoint that allows Asset to be viewed or edited.
     """
     model = Asset
-    filter_fields = (
-        "hostname", "ip", "systemuser__id", "admin_user__id", "platform__base",
-        "is_active", 'ip'
-    )
+    filterset_fields = {
+        'hostname': ['exact'],
+        'ip': ['exact'],
+        'systemuser__id': ['exact'],
+        'admin_user__id': ['exact'],
+        'platform__base': ['exact'],
+        'is_active': ['exact'],
+        'protocols': ['exact', 'icontains']
+    }
     search_fields = ("hostname", "ip")
     ordering_fields = ("hostname", "ip", "port", "cpu_cores")
     serializer_classes = {
@@ -41,7 +46,7 @@ class AssetViewSet(OrgBulkModelViewSet):
         'display': serializers.AssetDisplaySerializer,
     }
     permission_classes = (IsOrgAdminOrAppUser,)
-    extra_filter_backends = [AssetByNodeFilterBackend, LabelFilterBackend, IpInFilterBackend]
+    extra_filter_backends = [FilterAssetByNodeFilterBackend, LabelFilterBackend, IpInFilterBackend]
 
     def set_assets_node(self, assets):
         if not isinstance(assets, list):
@@ -74,7 +79,7 @@ class AssetPlatformViewSet(ModelViewSet):
     queryset = Platform.objects.all()
     permission_classes = (IsSuperUser,)
     serializer_class = serializers.PlatformSerializer
-    filter_fields = ['name', 'base']
+    filterset_fields = ['name', 'base']
     search_fields = ['name']
 
     def get_permissions(self):
@@ -90,32 +95,43 @@ class AssetPlatformViewSet(ModelViewSet):
         return super().check_object_permissions(request, obj)
 
 
-class AssetTaskCreateApi(generics.CreateAPIView):
+class AssetsTaskMixin:
+    def perform_assets_task(self, serializer):
+        data = serializer.validated_data
+        assets = data['assets']
+        action = data['action']
+        if action == "refresh":
+            task = update_assets_hardware_info_manual.delay(assets)
+        else:
+            task = test_assets_connectivity_manual.delay(assets)
+        data = getattr(serializer, '_data', {})
+        data["task"] = task.id
+        setattr(serializer, '_data', data)
+
+    def perform_create(self, serializer):
+        self.perform_assets_task(serializer)
+
+
+class AssetTaskCreateApi(AssetsTaskMixin, generics.CreateAPIView):
     model = Asset
     serializer_class = serializers.AssetTaskSerializer
     permission_classes = (IsOrgAdmin,)
 
-    def get_object(self):
-        pk = self.kwargs.get("pk")
-        instance = get_object_or_404(Asset, pk=pk)
-        return instance
+    def create(self, request, *args, **kwargs):
+        pk = self.kwargs.get('pk')
+        request.data['assets'] = [pk]
+        return super().create(request, *args, **kwargs)
 
-    def perform_create(self, serializer):
-        asset = self.get_object()
-        action = serializer.validated_data["action"]
-        if action == "refresh":
-            task = update_asset_hardware_info_manual.delay(asset)
-        else:
-            task = test_asset_connectivity_manual.delay(asset)
-        data = getattr(serializer, '_data', {})
-        data["task"] = task.id
-        setattr(serializer, '_data', data)
+
+class AssetsTaskCreateApi(AssetsTaskMixin, generics.CreateAPIView):
+    model = Asset
+    serializer_class = serializers.AssetTaskSerializer
+    permission_classes = (IsOrgAdmin,)
 
 
 class AssetGatewayListApi(generics.ListAPIView):
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.GatewayWithAuthSerializer
-    model = Asset
 
     def get_queryset(self):
         asset_id = self.kwargs.get('pk')

apps/assets/api/asset_user.py

@@ -10,10 +10,10 @@ from common.permissions import IsOrgAdminOrAppUser, NeedMFAVerify
 from common.utils import get_object_or_none, get_logger
 from common.mixins import CommonApiMixin
 from ..backends import AssetUserManager
-from ..models import Asset, Node, SystemUser
+from ..models import Node
 from .. import serializers
 from ..tasks import (
-    test_asset_users_connectivity_manual, push_system_user_a_asset_manual
+    test_asset_users_connectivity_manual
 )
 
 
@@ -28,7 +28,7 @@ logger = get_logger(__name__)
 class AssetUserFilterBackend(filters.BaseFilterBackend):
     def filter_queryset(self, request, queryset, view):
         kwargs = {}
-        for field in view.filter_fields:
+        for field in view.filterset_fields:
             value = request.GET.get(field)
             if not value:
                 continue
@@ -78,7 +78,7 @@ class AssetUserViewSet(CommonApiMixin, BulkModelViewSet):
         'retrieve': serializers.AssetUserReadSerializer,
     }
     permission_classes = [IsOrgAdminOrAppUser]
-    filter_fields = [
+    filterset_fields = [
         "id", "ip", "hostname", "username",
         "asset_id", "node_id",
         "prefer", "prefer_id",
@@ -100,12 +100,6 @@ class AssetUserViewSet(CommonApiMixin, BulkModelViewSet):
         obj = queryset.get(id=pk)
         return obj
 
-    def get_exception_handler(self):
-        def handler(e, context):
-            logger.error(e, exc_info=True)
-            return Response({"error": str(e)}, status=400)
-        return handler
-
     def perform_destroy(self, instance):
         manager = AssetUserManager()
         manager.delete(instance)
@@ -131,7 +125,7 @@ class AssetUserTaskCreateAPI(generics.CreateAPIView):
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.AssetUserTaskSerializer
     filter_backends = AssetUserViewSet.filter_backends
-    filter_fields = AssetUserViewSet.filter_fields
+    filterset_fields = AssetUserViewSet.filterset_fields
 
     def get_asset_users(self):
         manager = AssetUserManager()

apps/assets/api/cmd_filter.py

@@ -1,29 +1,39 @@
 # -*- coding: utf-8 -*-
 #
 
+from rest_framework.response import Response
+from rest_framework.generics import CreateAPIView, RetrieveDestroyAPIView
 from django.shortcuts import get_object_or_404
 
+from common.utils import reverse
+from common.utils import lazyproperty
 from orgs.mixins.api import OrgBulkModelViewSet
-from ..hands import IsOrgAdmin
+from orgs.utils import tmp_to_root_org
+from tickets.models import Ticket
+from tickets.api import GenericTicketStatusRetrieveCloseAPI
+from ..hands import IsOrgAdmin, IsAppUser
 from ..models import CommandFilter, CommandFilterRule
 from .. import serializers
 
 
-__all__ = ['CommandFilterViewSet', 'CommandFilterRuleViewSet']
+__all__ = [
+    'CommandFilterViewSet', 'CommandFilterRuleViewSet', 'CommandConfirmAPI',
+    'CommandConfirmStatusAPI'
+]
 
 
 class CommandFilterViewSet(OrgBulkModelViewSet):
     model = CommandFilter
-    filter_fields = ("name",)
-    search_fields = filter_fields
+    filterset_fields = ("name",)
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.CommandFilterSerializer
 
 
 class CommandFilterRuleViewSet(OrgBulkModelViewSet):
     model = CommandFilterRule
-    filter_fields = ("content",)
-    search_fields = filter_fields
+    filterset_fields = ("content",)
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.CommandFilterRuleSerializer
 
@@ -35,3 +45,50 @@ class CommandFilterRuleViewSet(OrgBulkModelViewSet):
         return cmd_filter.rules.all()
 
 
+class CommandConfirmAPI(CreateAPIView):
+    permission_classes = (IsAppUser, )
+    serializer_class = serializers.CommandConfirmSerializer
+
+    def create(self, request, *args, **kwargs):
+        ticket = self.create_command_confirm_ticket()
+        response_data = self.get_response_data(ticket)
+        return Response(data=response_data, status=200)
+
+    def create_command_confirm_ticket(self):
+        ticket = self.serializer.cmd_filter_rule.create_command_confirm_ticket(
+            run_command=self.serializer.data.get('run_command'),
+            session=self.serializer.session,
+            cmd_filter_rule=self.serializer.cmd_filter_rule,
+            org_id=self.serializer.org.id
+        )
+        return ticket
+
+    @staticmethod
+    def get_response_data(ticket):
+        confirm_status_url = reverse(
+            view_name='api-assets:command-confirm-status',
+            kwargs={'pk': str(ticket.id)}
+        )
+        ticket_detail_url = reverse(
+            view_name='api-tickets:ticket-detail',
+            kwargs={'pk': str(ticket.id)},
+            external=True, api_to_ui=True
+        )
+        ticket_detail_url = '{url}?type={type}'.format(url=ticket_detail_url, type=ticket.type)
+        return {
+            'check_confirm_status': {'method': 'GET', 'url': confirm_status_url},
+            'close_confirm': {'method': 'DELETE', 'url': confirm_status_url},
+            'ticket_detail_url': ticket_detail_url,
+            'reviewers': [str(user) for user in ticket.assignees.all()]
+        }
+
+    @lazyproperty
+    def serializer(self):
+        serializer = self.get_serializer(data=self.request.data)
+        serializer.is_valid(raise_exception=True)
+        return serializer
+
+
+class CommandConfirmStatusAPI(GenericTicketStatusRetrieveCloseAPI):
+    pass
+

apps/assets/api/domain.py

@@ -1,7 +1,9 @@
 # ~*~ coding: utf-8 ~*~
 
-from rest_framework.views import APIView, Response
 from django.views.generic.detail import SingleObjectMixin
+from django.utils.translation import ugettext as _
+from rest_framework.views import APIView, Response
+from rest_framework.serializers import ValidationError
 
 from common.utils import get_logger
 from common.permissions import IsOrgAdmin, IsOrgAdminOrAppUser
@@ -16,8 +18,8 @@ __all__ = ['DomainViewSet', 'GatewayViewSet', "GatewayTestConnectionApi"]
 
 class DomainViewSet(OrgBulkModelViewSet):
     model = Domain
-    filter_fields = ("name", )
-    search_fields = filter_fields
+    filterset_fields = ("name", )
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.DomainSerializer
 
@@ -29,7 +31,7 @@ class DomainViewSet(OrgBulkModelViewSet):
 
 class GatewayViewSet(OrgBulkModelViewSet):
     model = Gateway
-    filter_fields = ("domain__name", "name", "username", "ip", "domain")
+    filterset_fields = ("domain__name", "name", "username", "ip", "domain")
     search_fields = ("domain__name", "name", "username", "ip")
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.GatewaySerializer
@@ -42,6 +44,10 @@ class GatewayTestConnectionApi(SingleObjectMixin, APIView):
     def post(self, request, *args, **kwargs):
         self.object = self.get_object(Gateway.objects.all())
         local_port = self.request.data.get('port') or self.object.port
+        try:
+            local_port = int(local_port)
+        except ValueError:
+            raise ValidationError({'port': _('Number required')})
         ok, e = self.object.test_connective(local_port=local_port)
         if ok:
             return Response("ok")

apps/assets/api/favorite_asset.py

@@ -13,7 +13,7 @@ __all__ = ['FavoriteAssetViewSet']
 class FavoriteAssetViewSet(BulkModelViewSet):
     serializer_class = FavoriteAssetSerializer
     permission_classes = (IsValidUser,)
-    filter_fields = ['asset']
+    filterset_fields = ['asset']
 
     def dispatch(self, request, *args, **kwargs):
         with tmp_to_root_org():

apps/assets/api/gathered_user.py

@@ -18,5 +18,5 @@ class GatheredUserViewSet(OrgModelViewSet):
     permission_classes = [IsOrgAdmin]
     extra_filter_backends = [AssetRelatedByNodeFilterBackend]
 
-    filter_fields = ['asset', 'username', 'present', 'asset__ip', 'asset__hostname', 'asset_id']
+    filterset_fields = ['asset', 'username', 'present', 'asset__ip', 'asset__hostname', 'asset_id']
     search_fields = ['username', 'asset__ip', 'asset__hostname']

apps/assets/api/label.py

@@ -28,8 +28,8 @@ __all__ = ['LabelViewSet']
 
 class LabelViewSet(OrgBulkModelViewSet):
     model = Label
-    filter_fields = ("name", "value")
-    search_fields = filter_fields
+    filterset_fields = ("name", "value")
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.LabelSerializer
 

apps/assets/api/mixin.py

@@ -0,0 +1,92 @@
+from typing import List
+
+from common.utils.common import timeit
+from assets.models import Node, Asset
+from assets.pagination import NodeAssetTreePagination
+from common.utils import lazyproperty
+from assets.utils import get_node, is_query_node_all_assets
+
+
+class SerializeToTreeNodeMixin:
+
+    @timeit
+    def serialize_nodes(self, nodes: List[Node], with_asset_amount=False):
+        if with_asset_amount:
+            def _name(node: Node):
+                return '{} ({})'.format(node.value, node.assets_amount)
+        else:
+            def _name(node: Node):
+                return node.value
+        data = [
+            {
+                'id': node.key,
+                'name': _name(node),
+                'title': _name(node),
+                'pId': node.parent_key,
+                'isParent': True,
+                'open': node.is_org_root(),
+                'meta': {
+                    'node': {
+                        "id": node.id,
+                        "key": node.key,
+                        "value": node.value,
+                    },
+                    'type': 'node'
+                }
+            }
+            for node in nodes
+        ]
+        return data
+
+    def get_platform(self, asset: Asset):
+        default = 'file'
+        icon = {'windows', 'linux'}
+        platform = asset.platform_base.lower()
+        if platform in icon:
+            return platform
+        return default
+
+    @timeit
+    def serialize_assets(self, assets, node_key=None):
+        if node_key is None:
+            get_pid = lambda asset: getattr(asset, 'parent_key', '')
+        else:
+            get_pid = lambda asset: node_key
+
+        data = [
+            {
+                'id': str(asset.id),
+                'name': asset.hostname,
+                'title': asset.ip,
+                'pId': get_pid(asset),
+                'isParent': False,
+                'open': False,
+                'iconSkin': self.get_platform(asset),
+                'chkDisabled': not asset.is_active,
+                'meta': {
+                    'type': 'asset',
+                    'asset': {
+                        'id': asset.id,
+                        'hostname': asset.hostname,
+                        'ip': asset.ip,
+                        'protocols': asset.protocols_as_list,
+                        'platform': asset.platform_base,
+                        'org_name': asset.org_name
+                    },
+                }
+            }
+            for asset in assets
+        ]
+        return data
+
+
+class FilterAssetByNodeMixin:
+    pagination_class = NodeAssetTreePagination
+
+    @lazyproperty
+    def is_query_node_all_assets(self):
+        return is_query_node_all_assets(self.request)
+
+    @lazyproperty
+    def node(self):
+        return get_node(self.request)

apps/assets/api/node.py

@@ -1,29 +1,40 @@
 # ~*~ coding: utf-8 ~*~
+from functools import partial
+from collections import namedtuple, defaultdict
 
-from collections import namedtuple
 from rest_framework import status
 from rest_framework.serializers import ValidationError
 from rest_framework.response import Response
+from rest_framework.decorators import action
 from django.utils.translation import ugettext_lazy as _
 from django.shortcuts import get_object_or_404, Http404
+from django.db.models.signals import m2m_changed
 
+from common.const.http import POST
+from common.exceptions import SomeoneIsDoingThis
+from common.const.signals import PRE_REMOVE, POST_REMOVE
+from assets.models import Asset
 from common.utils import get_logger, get_object_or_none
 from common.tree import TreeNodeSerializer
 from orgs.mixins.api import OrgModelViewSet
 from orgs.mixins import generics
+from orgs.utils import current_org
 from ..hands import IsOrgAdmin
 from ..models import Node
 from ..tasks import (
     update_node_assets_hardware_info_manual,
     test_node_assets_connectivity_manual,
+    check_node_assets_amount_task
 )
 from .. import serializers
+from .mixin import SerializeToTreeNodeMixin
+from assets.locks import NodeAddChildrenLock
 
 
 logger = get_logger(__file__)
 __all__ = [
     'NodeViewSet', 'NodeChildrenApi', 'NodeAssetsApi',
-    'NodeAddAssetsApi', 'NodeRemoveAssetsApi', 'NodeReplaceAssetsApi',
+    'NodeAddAssetsApi', 'NodeRemoveAssetsApi', 'MoveAssetsToNodeApi',
     'NodeAddChildrenApi', 'NodeListAsTreeApi',
     'NodeChildrenAsTreeApi',
     'NodeTaskCreateApi',
@@ -32,7 +43,7 @@ __all__ = [
 
 class NodeViewSet(OrgModelViewSet):
     model = Node
-    filter_fields = ('value', 'key', 'id')
+    filterset_fields = ('value', 'key', 'id')
     search_fields = ('value', )
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.NodeSerializer
@@ -43,6 +54,11 @@ class NodeViewSet(OrgModelViewSet):
         serializer.validated_data["key"] = child_key
         serializer.save()
 
+    @action(methods=[POST], detail=False, url_path='check_assets_amount_task')
+    def check_assets_amount_task(self, request):
+        task = check_node_assets_amount_task.delay(current_org.id)
+        return Response(data={'task': task.id})
+
     def perform_update(self, serializer):
         node = self.get_object()
         if node.is_org_root() and node.value != serializer.validated_data['value']:
@@ -52,8 +68,11 @@ class NodeViewSet(OrgModelViewSet):
 
     def destroy(self, request, *args, **kwargs):
         node = self.get_object()
-        if node.has_children_or_has_assets():
-            error = _("Deletion failed and the node contains children or assets")
+        if node.is_org_root():
+            error = _("You can't delete the root node ({})".format(node.value))
+            return Response(data={'error': error}, status=status.HTTP_403_FORBIDDEN)
+        if node.has_offspring_assets():
+            error = _("Deletion failed and the node contains assets")
             return Response(data={'error': error}, status=status.HTTP_403_FORBIDDEN)
         return super().destroy(request, *args, **kwargs)
 
@@ -96,22 +115,27 @@ class NodeChildrenApi(generics.ListCreateAPIView):
         return super().initial(request, *args, **kwargs)
 
     def perform_create(self, serializer):
-        data = serializer.validated_data
-        _id = data.get("id")
-        value = data.get("value")
-        if not value:
-            value = self.instance.get_next_child_preset_name()
-        node = self.instance.create_child(value=value, _id=_id)
-        # 避免查询 full value
-        node._full_value = node.value
-        serializer.instance = node
+        with NodeAddChildrenLock(self.instance):
+            data = serializer.validated_data
+            _id = data.get("id")
+            value = data.get("value")
+            if not value:
+                value = self.instance.get_next_child_preset_name()
+            node = self.instance.create_child(value=value, _id=_id)
+            # 避免查询 full value
+            node._full_value = node.value
+            serializer.instance = node
 
     def get_object(self):
         pk = self.kwargs.get('pk') or self.request.query_params.get('id')
         key = self.request.query_params.get("key")
+
         if not pk and not key:
-            node = Node.org_root()
             self.is_initial = True
+            if current_org.is_root():
+                node = None
+            else:
+                node = Node.org_root()
             return node
         if pk:
             node = get_object_or_404(Node, pk=pk)
@@ -119,16 +143,26 @@ class NodeChildrenApi(generics.ListCreateAPIView):
             node = get_object_or_404(Node, key=key)
         return node
 
+    def get_org_root_queryset(self, query_all):
+        if query_all:
+            return Node.objects.all()
+        else:
+            return Node.org_root_nodes()
+
     def get_queryset(self):
         query_all = self.request.query_params.get("all", "0") == "all"
-        if not self.instance:
-            return Node.objects.none()
+
+        if self.is_initial and current_org.is_root():
+            return self.get_org_root_queryset(query_all)
 
         if self.is_initial:
             with_self = True
         else:
             with_self = False
 
+        if not self.instance:
+            return Node.objects.none()
+
         if query_all:
             queryset = self.instance.get_all_children(with_self=with_self)
         else:
@@ -136,7 +170,7 @@ class NodeChildrenApi(generics.ListCreateAPIView):
         return queryset
 
 
-class NodeChildrenAsTreeApi(NodeChildrenApi):
+class NodeChildrenAsTreeApi(SerializeToTreeNodeMixin, NodeChildrenApi):
     """
     节点子节点作为树返回，
     [
@@ -150,31 +184,23 @@ class NodeChildrenAsTreeApi(NodeChildrenApi):
 
     """
     model = Node
-    serializer_class = TreeNodeSerializer
-    http_method_names = ['get']
 
-    def get_queryset(self):
-        queryset = super().get_queryset()
-        queryset = [node.as_tree_node() for node in queryset]
-        queryset = self.add_assets_if_need(queryset)
-        queryset = sorted(queryset)
-        return queryset
+    def list(self, request, *args, **kwargs):
+        nodes = self.get_queryset().order_by('value')
+        nodes = self.serialize_nodes(nodes, with_asset_amount=True)
+        assets = self.get_assets()
+        data = [*nodes, *assets]
+        return Response(data=data)
 
-    def add_assets_if_need(self, queryset):
+    def get_assets(self):
         include_assets = self.request.query_params.get('assets', '0') == '1'
-        if not include_assets:
-            return queryset
+        if not self.instance or not include_assets:
+            return []
         assets = self.instance.get_assets().only(
-            "id", "hostname", "ip", "os",
-            "org_id", "protocols",
-        )
-        for asset in assets:
-            queryset.append(asset.as_tree_node(self.instance))
-        return queryset
-
-    def check_need_refresh_nodes(self):
-        if self.request.query_params.get('refresh', '0') == '1':
-            Node.refresh_nodes()
+            "id", "hostname", "ip", "os", "platform_id",
+            "org_id", "protocols", "is_active",
+        ).prefetch_related('platform')
+        return self.serialize_assets(assets, self.instance.key)
 
 
 class NodeAssetsApi(generics.ListAPIView):
@@ -197,13 +223,12 @@ class NodeAddChildrenApi(generics.UpdateAPIView):
     serializer_class = serializers.NodeAddChildrenSerializer
     instance = None
 
-    def put(self, request, *args, **kwargs):
+    def update(self, request, *args, **kwargs):
+        """ 同时支持 put 和 patch 方法"""
         instance = self.get_object()
-        nodes_id = request.data.get("nodes")
-        children = [get_object_or_none(Node, id=pk) for pk in nodes_id]
+        node_ids = request.data.get("nodes")
+        children = Node.objects.filter(id__in=node_ids)
         for node in children:
-            if not node:
-                continue
             node.parent = instance
         return Response("OK")
 
@@ -228,15 +253,18 @@ class NodeRemoveAssetsApi(generics.UpdateAPIView):
 
     def perform_update(self, serializer):
         assets = serializer.validated_data.get('assets')
-        instance = self.get_object()
-        if instance != Node.org_root():
-            instance.assets.remove(*tuple(assets))
-        else:
-            assets = [asset for asset in assets if asset.nodes.count() > 1]
-            instance.assets.remove(*tuple(assets))
+        node = self.get_object()
+        node.assets.remove(*assets)
+
+        # 把孤儿资产添加到 root 节点
+        orphan_assets = Asset.objects.filter(
+            id__in=[a.id for a in assets],
+            nodes__isnull=True
+        ).distinct()
+        Node.org_root().assets.add(*orphan_assets)
 
 
-class NodeReplaceAssetsApi(generics.UpdateAPIView):
+class MoveAssetsToNodeApi(generics.UpdateAPIView):
     model = Node
     serializer_class = serializers.NodeAssetsSerializer
     permission_classes = (IsOrgAdmin,)
@@ -244,9 +272,39 @@ class NodeReplaceAssetsApi(generics.UpdateAPIView):
 
     def perform_update(self, serializer):
         assets = serializer.validated_data.get('assets')
-        instance = self.get_object()
-        for asset in assets:
-            asset.nodes.set([instance])
+        node = self.get_object()
+        self.remove_old_nodes(assets)
+        node.assets.add(*assets)
+
+    def remove_old_nodes(self, assets):
+        m2m_model = Asset.nodes.through
+
+        # 查询资产与节点关系表，查出要移动资产与节点的所有关系
+        relates = m2m_model.objects.filter(asset__in=assets).values_list('asset_id', 'node_id')
+        if relates:
+            # 对关系以资产进行分组，用来发 `reverse=False` 信号
+            asset_nodes_mapper = defaultdict(set)
+            for asset_id, node_id in relates:
+                asset_nodes_mapper[asset_id].add(node_id)
+
+            # 组建一个资产 id -> Asset 的 mapper
+            asset_mapper = {asset.id: asset for asset in assets}
+
+            # 创建删除关系信号发送函数
+            senders = []
+            for asset_id, node_id_set in asset_nodes_mapper.items():
+                senders.append(partial(m2m_changed.send, sender=m2m_model, instance=asset_mapper[asset_id],
+                                       reverse=False, model=Node, pk_set=node_id_set))
+            # 发送 pre 信号
+            [sender(action=PRE_REMOVE) for sender in senders]
+            num = len(relates)
+            asset_ids, node_ids = zip(*relates)
+            # 删除之前的关系
+            rows, _i = m2m_model.objects.filter(asset_id__in=asset_ids, node_id__in=node_ids).delete()
+            if rows != num:
+                raise SomeoneIsDoingThis
+            # 发送 post 信号
+            [sender(action=POST_REMOVE) for sender in senders]
 
 
 class NodeTaskCreateApi(generics.CreateAPIView):
@@ -267,7 +325,6 @@ class NodeTaskCreateApi(generics.CreateAPIView):
 
     @staticmethod
     def refresh_nodes_cache():
-        Node.refresh_nodes()
         Task = namedtuple('Task', ['id'])
         task = Task(id="0")
         return task

apps/assets/api/system_user.py

@@ -3,23 +3,24 @@ from django.shortcuts import get_object_or_404
 from rest_framework.response import Response
 
 from common.utils import get_logger
-from common.permissions import IsOrgAdmin, IsOrgAdminOrAppUser, IsAppUser
+from common.permissions import IsOrgAdmin, IsOrgAdminOrAppUser, IsValidUser
 from orgs.mixins.api import OrgBulkModelViewSet
 from orgs.mixins import generics
-from orgs.utils import tmp_to_org
+from orgs.utils import tmp_to_root_org
 from ..models import SystemUser, Asset
 from .. import serializers
-from ..serializers import SystemUserWithAuthInfoSerializer
+from ..serializers import SystemUserWithAuthInfoSerializer, SystemUserTempAuthSerializer
 from ..tasks import (
     push_system_user_to_assets_manual, test_system_user_connectivity_manual,
-    push_system_user_a_asset_manual,
+    push_system_user_to_assets
 )
 
 
 logger = get_logger(__file__)
 __all__ = [
     'SystemUserViewSet', 'SystemUserAuthInfoApi', 'SystemUserAssetAuthInfoApi',
-    'SystemUserCommandFilterRuleListApi', 'SystemUserTaskApi',
+    'SystemUserCommandFilterRuleListApi', 'SystemUserTaskApi', 'SystemUserAssetsListView',
+    'SystemUserTempAuthInfoApi', 'SystemUserAppAuthInfoApi',
 ]
 
 
@@ -28,8 +29,12 @@ class SystemUserViewSet(OrgBulkModelViewSet):
     System user api set, for add,delete,update,list,retrieve resource
     """
     model = SystemUser
-    filter_fields = ("name", "username", "protocol")
-    search_fields = filter_fields
+    filterset_fields = {
+        'name': ['exact'],
+        'username': ['exact'],
+        'protocol': ['exact', 'in']
+    }
+    search_fields = filterset_fields
     serializer_class = serializers.SystemUserSerializer
     serializer_classes = {
         'default': serializers.SystemUserSerializer,
@@ -52,6 +57,25 @@ class SystemUserAuthInfoApi(generics.RetrieveUpdateDestroyAPIView):
         return Response(status=204)
 
 
+class SystemUserTempAuthInfoApi(generics.CreateAPIView):
+    model = SystemUser
+    permission_classes = (IsValidUser,)
+    serializer_class = SystemUserTempAuthSerializer
+
+    def create(self, request, *args, **kwargs):
+        serializer = super().get_serializer(data=request.data)
+        serializer.is_valid(raise_exception=True)
+        pk = kwargs.get('pk')
+        user = self.request.user
+        data = serializer.validated_data
+        instance_id = data.get('instance_id')
+
+        with tmp_to_root_org():
+            instance = get_object_or_404(SystemUser, pk=pk)
+            instance.set_temp_auth(instance_id, user, data)
+        return Response(serializer.data, status=201)
+
+
 class SystemUserAssetAuthInfoApi(generics.RetrieveAPIView):
     """
     Get system user with asset auth info
@@ -60,41 +84,49 @@ class SystemUserAssetAuthInfoApi(generics.RetrieveAPIView):
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = SystemUserWithAuthInfoSerializer
 
-    def get_exception_handler(self):
-        def handler(e, context):
-            return Response({"error": str(e)}, status=400)
-        return handler
+    def get_object(self):
+        instance = super().get_object()
+        asset_id = self.kwargs.get('asset_id')
+        user_id = self.request.query_params.get("user_id")
+        username = self.request.query_params.get("username")
+        instance.load_asset_more_auth(asset_id=asset_id, user_id=user_id, username=username)
+        return instance
+
+
+class SystemUserAppAuthInfoApi(generics.RetrieveAPIView):
+    """
+    Get system user with asset auth info
+    """
+    model = SystemUser
+    permission_classes = (IsOrgAdminOrAppUser,)
+    serializer_class = SystemUserWithAuthInfoSerializer
 
     def get_object(self):
         instance = super().get_object()
-        username = instance.username
-        if instance.username_same_with_user:
-            username = self.request.query_params.get("username")
-        asset_id = self.kwargs.get('aid')
-        asset = get_object_or_404(Asset, pk=asset_id)
-
-        with tmp_to_org(asset.org_id):
-            instance.load_asset_special_auth(asset=asset, username=username)
-            return instance
+        app_id = self.kwargs.get('app_id')
+        user_id = self.request.query_params.get("user_id")
+        if user_id:
+            instance.load_app_more_auth(app_id, user_id)
+        return instance
 
 
 class SystemUserTaskApi(generics.CreateAPIView):
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.SystemUserTaskSerializer
 
-    def do_push(self, system_user, asset=None):
-        if asset is None:
+    def do_push(self, system_user, asset_ids=None):
+        if asset_ids is None:
             task = push_system_user_to_assets_manual.delay(system_user)
         else:
             username = self.request.query_params.get('username')
-            task = push_system_user_a_asset_manual.delay(
-                system_user, asset, username=username
+            task = push_system_user_to_assets.delay(
+                system_user.id, asset_ids, username=username
             )
         return task
 
     @staticmethod
-    def do_test(system_user, asset=None):
-        task = test_system_user_connectivity_manual.delay(system_user)
+    def do_test(system_user, asset_ids):
+        task = test_system_user_connectivity_manual.delay(system_user, asset_ids)
         return task
 
     def get_object(self):
@@ -104,11 +136,20 @@ class SystemUserTaskApi(generics.CreateAPIView):
     def perform_create(self, serializer):
         action = serializer.validated_data["action"]
         asset = serializer.validated_data.get('asset')
+
+        if asset:
+            assets = [asset]
+        else:
+            assets = serializer.validated_data.get('assets') or []
+
+        asset_ids = [asset.id for asset in assets]
+        asset_ids = asset_ids if asset_ids else None
+
         system_user = self.get_object()
         if action == 'push':
-            task = self.do_push(system_user, asset)
+            task = self.do_push(system_user, asset_ids)
         else:
-            task = self.do_test(system_user, asset)
+            task = self.do_test(system_user, asset_ids)
         data = getattr(serializer, '_data', {})
         data["task"] = task.id
         setattr(serializer, '_data', data)
@@ -125,3 +166,18 @@ class SystemUserCommandFilterRuleListApi(generics.ListAPIView):
         pk = self.kwargs.get('pk', None)
         system_user = get_object_or_404(SystemUser, pk=pk)
         return system_user.cmd_filter_rules
+
+
+class SystemUserAssetsListView(generics.ListAPIView):
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.AssetSimpleSerializer
+    filterset_fields = ("hostname", "ip")
+    search_fields = filterset_fields
+
+    def get_object(self):
+        pk = self.kwargs.get('pk')
+        return get_object_or_404(SystemUser, pk=pk)
+
+    def get_queryset(self):
+        system_user = self.get_object()
+        return system_user.get_all_assets()

apps/assets/api/system_user_relation.py

@@ -19,9 +19,10 @@ __all__ = [
 class RelationMixin:
     def get_queryset(self):
         queryset = self.model.objects.all()
-        org_id = current_org.org_id()
-        if org_id is not None:
+        if not current_org.is_root():
+            org_id = current_org.org_id()
             queryset = queryset.filter(systemuser__org_id=org_id)
+
         queryset = queryset.annotate(systemuser_display=Concat(
             F('systemuser__name'), Value('('), F('systemuser__username'),
             Value(')')
@@ -65,7 +66,7 @@ class SystemUserAssetRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserAssetRelationSerializer
     model = models.SystemUser.assets.through
     permission_classes = (IsOrgAdmin,)
-    filter_fields = [
+    filterset_fields = [
         'id', 'asset', 'systemuser',
     ]
     search_fields = [
@@ -91,11 +92,11 @@ class SystemUserNodeRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserNodeRelationSerializer
     model = models.SystemUser.nodes.through
     permission_classes = (IsOrgAdmin,)
-    filter_fields = [
+    filterset_fields = [
         'id', 'node', 'systemuser',
     ]
     search_fields = [
-        "node__value", "systemuser__name", "systemuser_username"
+        "node__value", "systemuser__name", "systemuser__username"
     ]
 
     def get_objects_attr(self):
@@ -112,7 +113,7 @@ class SystemUserUserRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserUserRelationSerializer
     model = models.SystemUser.users.through
     permission_classes = (IsOrgAdmin,)
-    filter_fields = [
+    filterset_fields = [
         'id', 'user', 'systemuser',
     ]
     search_fields = [

apps/assets/apps.py

@@ -1,16 +1,6 @@
 from __future__ import unicode_literals
 
 from django.apps import AppConfig
-from django.db.models.signals import post_migrate
-
-
-def initial_some_nodes():
-    from .models import Node
-    Node.initial_some_nodes()
-
-
-def initial_some_nodes_callback(sender, **kwargs):
-    initial_some_nodes()
 
 
 class AssetsConfig(AppConfig):
@@ -19,7 +9,3 @@ class AssetsConfig(AppConfig):
     def ready(self):
         super().ready()
         from . import signals_handler
-        try:
-            initial_some_nodes()
-        except Exception:
-            post_migrate.connect(initial_some_nodes_callback, sender=self)

apps/assets/backends/base.py

@@ -31,16 +31,16 @@ class BaseBackend:
     def qs_to_values(qs):
         values = qs.values(
             'hostname', 'ip', "asset_id",
-            'username', 'password', 'private_key', 'public_key',
+            'name', 'username', 'password', 'private_key', 'public_key',
             'score', 'version',
             "asset_username", "union_id",
             'date_created', 'date_updated',
-            'org_id', 'backend',
+            'org_id', 'backend', 'backend_display'
         )
         return values
 
     @staticmethod
-    def make_assets_as_id(assets):
+    def make_assets_as_ids(assets):
         if not assets:
             return []
         if isinstance(assets[0], Asset):

apps/assets/backends/db.py

@@ -4,6 +4,7 @@ from django.utils.translation import ugettext as _
 from functools import reduce
 from django.db.models import F, CharField, Value, IntegerField, Q, Count
 from django.db.models.functions import Concat
+from rest_framework.exceptions import PermissionDenied
 
 from common.utils import get_object_or_none
 from orgs.utils import current_org
@@ -69,9 +70,9 @@ class DBBackend(BaseBackend):
         self.queryset = self.queryset.filter(union_id=union_id)
 
     def _filter_assets(self, assets):
-        assets_id = self.make_assets_as_id(assets)
-        if assets_id:
-            self.queryset = self.queryset.filter(asset_id__in=assets_id)
+        asset_ids = self.make_assets_as_ids(assets)
+        if asset_ids:
+            self.queryset = self.queryset.filter(asset_id__in=asset_ids)
 
     def _filter_node(self, node):
         pass
@@ -106,6 +107,7 @@ class DBBackend(BaseBackend):
 class SystemUserBackend(DBBackend):
     model = SystemUser.assets.through
     backend = 'system_user'
+    backend_display = _('System user')
     prefer = backend
     base_score = 0
     union_id_length = 2
@@ -138,6 +140,7 @@ class SystemUserBackend(DBBackend):
         kwargs = dict(
             hostname=F("asset__hostname"),
             ip=F("asset__ip"),
+            name=F("systemuser__name"),
             username=F("systemuser__username"),
             password=F("systemuser__password"),
             private_key=F("systemuser__private_key"),
@@ -152,7 +155,8 @@ class SystemUserBackend(DBBackend):
             union_id=Concat(F("systemuser_id"), Value("_"), F("asset_id"),
                             output_field=CharField()),
             org_id=F("asset__org_id"),
-            backend=Value(self.backend, CharField())
+            backend=Value(self.backend, CharField()),
+            backend_display=Value(self.backend_display, CharField()),
         )
         return kwargs
 
@@ -165,7 +169,7 @@ class SystemUserBackend(DBBackend):
         kwargs = self.get_annotate()
         filters = self.get_filter()
         qs = self.model.objects.all().annotate(**kwargs)
-        if current_org.org_id() is not None:
+        if not current_org.is_root():
             filters['org_id'] = current_org.org_id()
         qs = qs.filter(**filters)
         qs = self.qs_to_values(qs)
@@ -174,12 +178,17 @@ class SystemUserBackend(DBBackend):
 
 class DynamicSystemUserBackend(SystemUserBackend):
     backend = 'system_user_dynamic'
+    backend_display = _('System user(Dynamic)')
     prefer = 'system_user'
     union_id_length = 3
 
     def get_annotate(self):
         kwargs = super().get_annotate()
         kwargs.update(dict(
+            name=Concat(
+                F("systemuser__users__name"), Value('('), F("systemuser__name"), Value(')'),
+                output_field=CharField()
+            ),
             username=F("systemuser__users__username"),
             asset_username=Concat(
                 F("asset__id"), Value("_"),
@@ -221,6 +230,7 @@ class DynamicSystemUserBackend(SystemUserBackend):
 class AdminUserBackend(DBBackend):
     model = Asset
     backend = 'admin_user'
+    backend_display = _('Admin user')
     prefer = backend
     base_score = 200
 
@@ -241,11 +251,12 @@ class AdminUserBackend(DBBackend):
         )
 
     def _perform_delete_by_union_id(self, union_id_cleaned):
-        raise PermissionError(_("Could not remove asset admin user"))
+        raise PermissionDenied(_("Could not remove asset admin user"))
 
     def all(self):
         qs = self.model.objects.all().annotate(
             asset_id=F("id"),
+            name=F("admin_user__name"),
             username=F("admin_user__username"),
             password=F("admin_user__password"),
             private_key=F("admin_user__private_key"),
@@ -256,6 +267,7 @@ class AdminUserBackend(DBBackend):
             asset_username=Concat(F("id"), Value("_"), F("admin_user__username"), output_field=CharField()),
             union_id=Concat(F("admin_user_id"), Value("_"), F("id"), output_field=CharField()),
             backend=Value(self.backend, CharField()),
+            backend_display=Value(self.backend_display, CharField()),
         )
         qs = self.qs_to_values(qs)
         return qs
@@ -264,6 +276,7 @@ class AdminUserBackend(DBBackend):
 class AuthbookBackend(DBBackend):
     model = AuthBook
     backend = 'db'
+    backend_display = _('Database')
     prefer = backend
     base_score = 400
 
@@ -302,7 +315,7 @@ class AuthbookBackend(DBBackend):
         authbook_id, asset_id = union_id_cleaned
         authbook = get_object_or_none(AuthBook, pk=authbook_id)
         if authbook.is_latest:
-            raise PermissionError(_("Latest version could not be delete"))
+            raise PermissionDenied(_("Latest version could not be delete"))
         AuthBook.objects.filter(id=authbook_id).delete()
 
     def all(self):
@@ -313,6 +326,7 @@ class AuthbookBackend(DBBackend):
             asset_username=Concat(F("asset__id"), Value("_"), F("username"), output_field=CharField()),
             union_id=Concat(F("id"), Value("_"), F("asset_id"), output_field=CharField()),
             backend=Value(self.backend, CharField()),
+            backend_display=Value(self.backend_display, CharField()),
         )
         qs = self.qs_to_values(qs)
         return qs

apps/assets/exceptions.py

@@ -0,0 +1,6 @@
+from rest_framework import status
+from common.exceptions import JMSException
+
+
+class NodeIsBeingUpdatedByOthers(JMSException):
+    status_code = status.HTTP_409_CONFLICT

apps/assets/filters.py

@@ -5,8 +5,8 @@ from rest_framework.compat import coreapi, coreschema
 from rest_framework import filters
 from django.db.models import Q
 
-from common.utils import dict_get_any, is_uuid, get_object_or_none
-from .models import Node, Label
+from .models import Label
+from assets.utils import is_query_node_all_assets, get_node
 
 
 class AssetByNodeFilterBackend(filters.BaseFilterBackend):
@@ -21,47 +21,54 @@ class AssetByNodeFilterBackend(filters.BaseFilterBackend):
             for field in self.fields
         ]
 
-    @staticmethod
-    def is_query_all(request):
-        query_all_arg = request.query_params.get('all')
-        show_current_asset_arg = request.query_params.get('show_current_asset')
+    def filter_node_related_all(self, queryset, node):
+        return queryset.filter(
+            Q(nodes__key__istartswith=f'{node.key}:') |
+            Q(nodes__key=node.key)
+        ).distinct()
 
-        query_all = query_all_arg == '1'
-        if show_current_asset_arg is not None:
-            query_all = show_current_asset_arg != '1'
-        return query_all
-
-    @staticmethod
-    def get_query_node(request):
-        node_id = dict_get_any(request.query_params, ['node', 'node_id'])
-        if not node_id:
-            return None, False
-
-        if is_uuid(node_id):
-            node = get_object_or_none(Node, id=node_id)
-        else:
-            node = get_object_or_none(Node, key=node_id)
-        return node, True
-
-    @staticmethod
-    def perform_query(pattern, queryset):
-        return queryset.filter(nodes__key__regex=pattern).distinct()
+    def filter_node_related_direct(self, queryset, node):
+        return queryset.filter(nodes__key=node.key).distinct()
 
     def filter_queryset(self, request, queryset, view):
-        node, has_query_arg = self.get_query_node(request)
-        if not has_query_arg:
-            return queryset
-
+        node = get_node(request)
         if node is None:
             return queryset
-        query_all = self.is_query_all(request)
+
+        query_all = is_query_node_all_assets(request)
         if query_all:
-            pattern = node.get_all_children_pattern(with_self=True)
+            return self.filter_node_related_all(queryset, node)
         else:
-            # pattern = node.get_children_key_pattern(with_self=True)
-            # 只显示当前节点下资产
-            pattern = r"^{}$".format(node.key)
-        return self.perform_query(pattern, queryset)
+            return self.filter_node_related_direct(queryset, node)
+
+
+class FilterAssetByNodeFilterBackend(filters.BaseFilterBackend):
+    """
+    需要与 `assets.api.mixin.FilterAssetByNodeMixin` 配合使用
+    """
+    fields = ['node', 'all']
+
+    def get_schema_fields(self, view):
+        return [
+            coreapi.Field(
+                name=field, location='query', required=False,
+                type='string', example='', description='', schema=None,
+            )
+            for field in self.fields
+        ]
+
+    def filter_queryset(self, request, queryset, view):
+        node = view.node
+        if node is None:
+            return queryset
+        query_all = view.is_query_node_all_assets
+        if query_all:
+            return queryset.filter(
+                Q(nodes__key__istartswith=f'{node.key}:') |
+                Q(nodes__key=node.key)
+            ).distinct()
+        else:
+            return queryset.filter(nodes__key=node.key).distinct()
 
 
 class LabelFilterBackend(filters.BaseFilterBackend):
@@ -113,9 +120,14 @@ class LabelFilterBackend(filters.BaseFilterBackend):
 
 
 class AssetRelatedByNodeFilterBackend(AssetByNodeFilterBackend):
-    @staticmethod
-    def perform_query(pattern, queryset):
-        return queryset.filter(asset__nodes__key__regex=pattern).distinct()
+    def filter_node_related_all(self, queryset, node):
+        return queryset.filter(
+            Q(asset__nodes__key__istartswith=f'{node.key}:') |
+            Q(asset__nodes__key=node.key)
+        ).distinct()
+
+    def filter_node_related_direct(self, queryset, node):
+        return queryset.filter(asset__nodes__key=node.key).distinct()
 
 
 class IpInFilterBackend(filters.BaseFilterBackend):

apps/assets/locks.py

@@ -0,0 +1,29 @@
+from orgs.utils import current_org
+from common.utils.lock import DistributedLock
+from assets.models import Node
+
+
+class NodeTreeUpdateLock(DistributedLock):
+    name_template = 'assets.node.tree.update.<org_id:{org_id}>'
+
+    def get_name(self):
+        if current_org:
+            org_id = current_org.id
+        else:
+            org_id = 'current_org_is_null'
+        name = self.name_template.format(
+            org_id=org_id
+        )
+        return name
+
+    def __init__(self):
+        name = self.get_name()
+        super().__init__(name=name, release_on_transaction_commit=True, reentrant=True)
+
+
+class NodeAddChildrenLock(DistributedLock):
+    name_template = 'assets.node.add_children.<org_id:{org_id}>'
+
+    def __init__(self, node: Node):
+        name = self.name_template.format(org_id=node.org_id)
+        super().__init__(name=name, release_on_transaction_commit=True)

apps/assets/migrations/0002_auto_20180105_1807.py

@@ -0,0 +1,35 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-01-05 10:07
+from __future__ import unicode_literals
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='adminuser',
+            options={'ordering': ['name'], 'verbose_name': 'Admin user'},
+        ),
+        migrations.AlterModelOptions(
+            name='asset',
+            options={'verbose_name': 'Asset'},
+        ),
+        migrations.AlterModelOptions(
+            name='assetgroup',
+            options={'ordering': ['name'], 'verbose_name': 'Asset group'},
+        ),
+        migrations.AlterModelOptions(
+            name='cluster',
+            options={'ordering': ['name'], 'verbose_name': 'Cluster'},
+        ),
+        migrations.AlterModelOptions(
+            name='systemuser',
+            options={'ordering': ['name'], 'verbose_name': 'System user'},
+        ),
+    ]

apps/assets/migrations/0003_auto_20180109_2331.py

@@ -0,0 +1,22 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-01-09 15:31
+from __future__ import unicode_literals
+
+import assets.models.asset
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0002_auto_20180105_1807'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='asset',
+            name='cluster',
+            field=models.ForeignKey(default=assets.models.asset.default_cluster, on_delete=django.db.models.deletion.SET_DEFAULT, related_name='assets', to='assets.Cluster', verbose_name='Cluster'),
+        ),
+    ]

apps/assets/migrations/0004_auto_20180125_1218.py

@@ -0,0 +1,20 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-01-25 04:18
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0003_auto_20180109_2331'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='assetgroup',
+            name='created_by',
+            field=models.CharField(blank=True, max_length=32, null=True, verbose_name='Created by'),
+        ),
+    ]

apps/assets/migrations/0005_auto_20180126_1637.py

@@ -0,0 +1,40 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-01-26 08:37
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0004_auto_20180125_1218'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Label',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(max_length=128, verbose_name='Name')),
+                ('value', models.CharField(max_length=128, verbose_name='Value')),
+                ('category', models.CharField(choices=[('S', 'System'), ('U', 'User')], default='U', max_length=128, verbose_name='Category')),
+                ('is_active', models.BooleanField(default=True, verbose_name='Is active')),
+                ('comment', models.TextField(blank=True, null=True, verbose_name='Comment')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+            ],
+            options={
+                'db_table': 'assets_label',
+            },
+        ),
+        migrations.AlterUniqueTogether(
+            name='label',
+            unique_together=set([('name', 'value')]),
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='labels',
+            field=models.ManyToManyField(blank=True, related_name='assets', to='assets.Label', verbose_name='Labels'),
+        ),
+    ]

apps/assets/migrations/0006_auto_20180130_1502.py

@@ -0,0 +1,39 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-01-30 07:02
+from __future__ import unicode_literals
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0005_auto_20180126_1637'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='asset',
+            name='cabinet_no',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='cabinet_pos',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='env',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='remote_card_ip',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='status',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='type',
+        ),
+    ]

apps/assets/migrations/0007_auto_20180225_1815.py

@@ -0,0 +1,60 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-02-25 10:15
+from __future__ import unicode_literals
+
+import assets.models.asset
+from django.db import migrations, models
+import django.db.models.deletion
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0006_auto_20180130_1502'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Node',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('key', models.CharField(max_length=64, unique=True, verbose_name='Key')),
+                ('value', models.CharField(max_length=128, unique=True, verbose_name='Value')),
+                ('child_mark', models.IntegerField(default=0)),
+                ('date_create', models.DateTimeField(auto_now_add=True)),
+            ],
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='cluster',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='groups',
+        ),
+        migrations.RemoveField(
+            model_name='systemuser',
+            name='cluster',
+        ),
+        migrations.AlterField(
+            model_name='asset',
+            name='admin_user',
+            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.PROTECT, to='assets.AdminUser', verbose_name='Admin user'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='protocol',
+            field=models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp')], default='ssh', max_length=16, verbose_name='Protocol'),
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='nodes',
+            field=models.ManyToManyField(default=assets.models.asset.default_node, related_name='assets', to='assets.Node', verbose_name='Nodes'),
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='nodes',
+            field=models.ManyToManyField(blank=True, to='assets.Node', verbose_name='Nodes'),
+        ),
+    ]

apps/assets/migrations/0008_auto_20180306_1804.py

@@ -0,0 +1,40 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-03-06 10:04
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0007_auto_20180225_1815'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='adminuser',
+            name='created_by',
+            field=models.CharField(max_length=128, null=True, verbose_name='Created by'),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='username',
+            field=models.CharField(max_length=128, verbose_name='Username'),
+        ),
+        migrations.AlterField(
+            model_name='asset',
+            name='platform',
+            field=models.CharField(choices=[('Linux', 'Linux'), ('Unix', 'Unix'), ('MacOS', 'MacOS'), ('BSD', 'BSD'), ('Windows', 'Windows'), ('Other', 'Other')], default='Linux', max_length=128, verbose_name='Platform'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='created_by',
+            field=models.CharField(max_length=128, null=True, verbose_name='Created by'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='username',
+            field=models.CharField(max_length=128, verbose_name='Username'),
+        ),
+    ]

apps/assets/migrations/0009_auto_20180307_1212.py

@@ -0,0 +1,20 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-03-07 04:12
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0008_auto_20180306_1804'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='node',
+            name='value',
+            field=models.CharField(max_length=128, verbose_name='Value'),
+        ),
+    ]

apps/assets/migrations/0010_auto_20180307_1749.py

@@ -0,0 +1,20 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-03-07 09:49
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0009_auto_20180307_1212'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='node',
+            name='value',
+            field=models.CharField(max_length=128, unique=True, verbose_name='Value'),
+        ),
+    ]

apps/assets/migrations/0011_auto_20180326_0957.py

@@ -0,0 +1,55 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-03-26 01:57
+from __future__ import unicode_literals
+
+import assets.models.utils
+from django.db import migrations, models
+import django.db.models.deletion
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0010_auto_20180307_1749'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Domain',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(max_length=128, unique=True, verbose_name='Name')),
+                ('comment', models.TextField(blank=True, verbose_name='Comment')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+            ],
+        ),
+        migrations.CreateModel(
+            name='Gateway',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(max_length=128, unique=True, verbose_name='Name')),
+                ('username', models.CharField(max_length=128, verbose_name='Username')),
+                ('_password', models.CharField(blank=True, max_length=256, null=True, verbose_name='Password')),
+                ('_private_key', models.TextField(blank=True, max_length=4096, null=True, validators=[assets.models.utils.private_key_validator], verbose_name='SSH private key')),
+                ('_public_key', models.TextField(blank=True, max_length=4096, verbose_name='SSH public key')),
+                ('date_created', models.DateTimeField(auto_now_add=True)),
+                ('date_updated', models.DateTimeField(auto_now=True)),
+                ('created_by', models.CharField(max_length=128, null=True, verbose_name='Created by')),
+                ('ip', models.GenericIPAddressField(db_index=True, verbose_name='IP')),
+                ('port', models.IntegerField(default=22, verbose_name='Port')),
+                ('protocol', models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp')], default='ssh', max_length=16, verbose_name='Protocol')),
+                ('comment', models.CharField(blank=True, max_length=128, null=True, verbose_name='Comment')),
+                ('is_active', models.BooleanField(default=True, verbose_name='Is active')),
+                ('domain', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='assets.Domain', verbose_name='Domain')),
+            ],
+            options={
+                'abstract': False,
+            },
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='domain',
+            field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='assets', to='assets.Domain', verbose_name='Domain'),
+        ),
+    ]

apps/assets/migrations/0012_auto_20180404_1302.py

@@ -0,0 +1,21 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11 on 2018-04-04 05:02
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0011_auto_20180326_0957'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='asset',
+            name='domain',
+            field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='assets', to='assets.Domain', verbose_name='Domain'),
+        ),
+    ]