Merge pull request #2893 from jumpserver/dev

Dev

.github/ISSUE_TEMPLATE.md

@@ -1,7 +1,8 @@
 [简述你的问题]
 
+
 ##### 使用版本
-[请提供你使用的Jumpserver版本 0.3.2 或 0.5.0]
+[请提供你使用的Jumpserver版本 1.x.x 注: 0.3.x不再提供支持]
 
 ##### 问题复现步骤
 1. [步骤1]

.gitignore

@@ -17,7 +17,7 @@ dump.rdb
 .idea/
 db.sqlite3
 config.py
-migrations/
+config.yml
 *.log
 host_rsa_key
 *.bat
@@ -32,3 +32,5 @@ django.db
 celerybeat-schedule.db
 data/static
 docs/_build/
+xpack
+logs/*

Dockerfile

@@ -0,0 +1,25 @@
+FROM registry.fit2cloud.com/public/python:v3
+MAINTAINER Jumpserver Team <ibuler@qq.com>
+
+WORKDIR /opt/jumpserver
+RUN useradd jumpserver
+
+COPY ./requirements /tmp/requirements
+
+RUN yum -y install epel-release && \
+      echo -e "[mysql]\nname=mysql\nbaseurl=https://mirrors.tuna.tsinghua.edu.cn/mysql/yum/mysql57-community-el6/\ngpgcheck=0\nenabled=1" > /etc/yum.repos.d/mysql.repo
+RUN cd /tmp/requirements && yum -y install $(cat rpm_requirements.txt)
+RUN cd /tmp/requirements && pip install --upgrade pip setuptools && \
+    pip install -i https://mirrors.aliyun.com/pypi/simple/ -r requirements.txt || pip install -r requirements.txt
+RUN mkdir -p /root/.ssh/ && echo -e "Host *\n\tStrictHostKeyChecking no\n\tUserKnownHostsFile /dev/null" > /root/.ssh/config
+
+COPY . /opt/jumpserver
+RUN echo > config.yml
+VOLUME /opt/jumpserver/data
+VOLUME /opt/jumpserver/logs
+
+ENV LANG=zh_CN.UTF-8
+ENV LC_ALL=zh_CN.UTF-8
+
+EXPOSE 8080
+ENTRYPOINT ["./entrypoint.sh"]

README.md

@@ -1,58 +1,213 @@
-## Jumpserver
+## Jumpserver 多云环境下更好用的堡垒机
 
 [![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
-[![Django](https://img.shields.io/badge/django-1.11-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
-[![Ansible](https://img.shields.io/badge/ansible-2.2.2.0-blue.svg?style=plastic)](https://www.ansible.com/)
-[![Paramiko](https://img.shields.io/badge/paramiko-2.1.2-green.svg?style=plastic)](http://www.paramiko.org/)
+[![Django](https://img.shields.io/badge/django-2.1-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
+[![Ansible](https://img.shields.io/badge/ansible-2.4.2.0-blue.svg?style=plastic)](https://www.ansible.com/)
+[![Paramiko](https://img.shields.io/badge/paramiko-2.4.1-green.svg?style=plastic)](http://www.paramiko.org/)
 
 
 ----
 
-Jumpserver是全球首款完全开源的堡垒机，使用GNU GPL v2.0开源协议，是符合 4A 的专业运维审计系统。
+Jumpserver 是全球首款完全开源的堡垒机，使用 GNU GPL v2.0 开源协议，是符合 4A 的专业运维审计系统。
 
-Jumpserver使用Python / Django 进行开发，遵循 Web 2.0 规范，配备了业界领先的 Web Terminal 解决方案，交互界面美观、用户体验好。
+Jumpserver 使用 Python / Django 进行开发，遵循 Web 2.0 规范，配备了业界领先的 Web Terminal 解决方案，交互界面美观、用户体验好。
 
-Jumpserver采纳分布式架构，支持多机房跨区域部署，中心节点提供 API，各机房部署登录节点，可横向扩展、无并发限制。
+Jumpserver 采纳分布式架构，支持多机房跨区域部署，中心节点提供 API，各机房部署登录节点，可横向扩展、无并发限制。
 
 改变世界，从一点点开始。
 
-----
+- [English Version](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
+
 
 ### 功能
-  - 统一认证
-  - 资产管理
-  - 统一授权
-  - 审计
-  - 支持LDAP认证
-  - Web terminal
-  - SSH Server
-  - 支持Windows RDP
+----
+
+<table class="subscription-level-table">
+    <tr class="subscription-level-tr-border">
+        <th  style="background-color: #1ab394;color: #ffffff;" colspan="3">Jumpserver提供的堡垒机必备功能</th>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-first-td-background-style" rowspan="4">身份验证 Authentication</td>
+        <td class="features-second-td-background-style" rowspan="3" >登录认证
+        </td>
+        <td class="features-third-td-background-style">资源统一登录和认证
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">LDAP认证
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">支持OpenID，实现单点登录
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-second-td-background-style">多因子认证
+        </td>
+        <td class="features-third-td-background-style">MFA（Google Authenticator）
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-first-td-background-style" rowspan="9">账号管理 Account</td>
+        <td class="features-second-td-background-style" rowspan="2">集中账号管理
+        </td>
+        <td class="features-third-td-background-style">管理用户管理
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">系统用户管理
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-second-td-background-style" rowspan="4">统一密码管理
+        </td>
+        <td class="features-third-td-background-style">资产密码托管
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">自动生成密码
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">密码自动推送
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">密码过期设置
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-outline-td-background-style" rowspan="2">批量密码变更(X-PACK)
+        </td>
+        <td class="features-outline-td-background-style">定期批量修改密码
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-outline-td-background-style">生成随机密码
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-outline-td-background-style">多云环境的资产纳管(X-PACK)
+        </td>
+        <td class="features-outline-td-background-style">对私有云、公有云资产统一纳管
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-first-td-background-style" rowspan="9">授权控制 Authorization</td>
+        <td class="features-second-td-background-style" rowspan="3">资产授权管理
+        </td>
+        <td class="features-third-td-background-style">资产树
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">资产或资产组灵活授权
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">节点内资产自动继承授权
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-outline-td-background-style">RemoteApp(X-PACK)
+        </td>
+        <td class="features-outline-td-background-style">实现更细粒度的应用级授权
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-outline-td-background-style">组织管理(X-PACK)
+        </td>
+        <td class="features-outline-td-background-style">实现多租户管理，权限隔离
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-second-td-background-style">多维度授权
+        </td>
+        <td class="features-third-td-background-style">可对用户、用户组或系统角色授权
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-second-td-background-style">指令限制
+        </td>
+        <td class="features-third-td-background-style">限制特权指令使用，支持黑白名单
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-second-td-background-style">统一文件传输
+        </td>
+        <td class="features-third-td-background-style">SFTP 文件上传/下载
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-second-td-background-style">文件管理
+        </td>
+        <td class="features-third-td-background-style">Web SFTP 文件管理
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-first-td-background-style" rowspan="6">安全审计 Audit</td>
+        <td class="features-second-td-background-style" rowspan="2">会话管理
+        </td>
+        <td class="features-third-td-background-style">在线会话管理
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">历史会话管理
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-second-td-background-style" rowspan="2">录像管理
+        </td>
+        <td class="features-third-td-background-style">Linux 录像支持
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-third-td-background-style">Windows 录像支持
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-second-td-background-style">指令审计
+        </td>
+        <td class="features-third-td-background-style">指令记录
+        </td>
+    </tr>
+    <tr class="subscription-level-tr-border">
+        <td class="features-second-td-background-style">文件传输审计
+        </td>
+        <td class="features-third-td-background-style">上传/下载记录审计
+        </td>
+    </tr>
+</table>
+
 
 ### 开始使用
+----
 
-快速开始文档  [Docker安装](http://docs.jumpserver.org/zh/latest/quickstart.html)
+- 快速开始文档  [Docker 安装](http://docs.jumpserver.org/zh/docs/dockerinstall.html)
 
-一步一步安装文档 [详细部署](http://docs.jumpserver.org/zh/latest/step_by_step.html)
+- Step by Step 安装文档 [详细部署](http://docs.jumpserver.org/zh/docs/step_by_step.html)
 
-也可以查看我们完整文档包括了使用和开发 [文档](http://docs.jumpserver.org)
+- 也可以查看我们完整文档 [文档](http://docs.jumpserver.org)
 
-### Demo 和 截图 
+### Demo、视频 和 截图
+----
 
-我们提供了DEMO和截图可以让你快速了解Jumpserver
+我们提供了 Demo 、演示视频和截图可以让你快速了解 Jumpserver
 
-[DEMO](http://demo.jumpserver.org)
-[截图](http://docs.jumpserver.org/zh/docs/snapshot.html)
+- [Demo](https://demo.jumpserver.org/auth/login/?next=/)
+- [视频](https://fit2cloud2-offline-installer.oss-cn-beijing.aliyuncs.com/tools/Jumpserver%20%E4%BB%8B%E7%BB%8Dv1.4.mp4)
+- [截图](http://docs.jumpserver.org/zh/docs/snapshot.html)
 
-### SDK 
+### SDK
+----
 
-我们还编写了一些SDK，供你其它系统快速和Jumpserver APi交互，
+我们还编写了一些SDK，供你的其它系统快速和 Jumpserver API 交互
 
-- [python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver其它组件使用这个SDK完成交互
-- [java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) 恺珺同学提供的Java版本的SDK
+- [Python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver其它组件使用这个SDK完成交互
+- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) 恺珺同学提供的Java版本的SDK
 
 
 ### License & Copyright
-Copyright (c) 2014-2018 Beijing Duizhan Tech, Inc., All rights reserved.
+Copyright (c) 2014-2019 飞致云 FIT2CLOUD, All rights reserved.
 
 Licensed under The GNU General Public License version 2 (GPLv2)  (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
 

README_EN.md

@@ -0,0 +1,58 @@
+## Jumpserver 
+
+[![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
+[![Django](https://img.shields.io/badge/django-2.1-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
+[![Ansible](https://img.shields.io/badge/ansible-2.4.2.0-blue.svg?style=plastic)](https://www.ansible.com/)
+[![Paramiko](https://img.shields.io/badge/paramiko-2.4.1-green.svg?style=plastic)](http://www.paramiko.org/)
+
+
+----
+
+- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
+
+Jumpserver is the first fully open source bastion in the world, based on the GNU GPL v2.0 open source protocol. Jumpserver is a  professional operation and maintenance audit system conforms to 4A specifications.
+
+Jumpserver is developed using Python / Django, conforms to the Web 2.0 specification, and is equipped with the industry-leading Web Terminal solution which have beautiful interface and great user experience.
+
+Jumpserver adopts a distributed architecture to support multi-branch deployment across multiple areas. The central node provides APIs, and login nodes are deployed in each branch. It can be scaled horizontally without concurrency restrictions.
+
+Change the world, starting from little things.
+
+----
+
+### Features
+
+ ![Jumpserver 功能](https://jumpserver-release.oss-cn-hangzhou.aliyuncs.com/Jumpserver148.jpeg "Jumpserver 功能")
+
+### Start 
+
+Quick start  [Docker Install](http://docs.jumpserver.org/zh/docs/dockerinstall.html)
+
+Step by Step deployment. [Docs](http://docs.jumpserver.org/zh/docs/step_by_step.html)
+
+Full documentation [Docs](http://docs.jumpserver.org)
+
+### Demo、Video 和 Snapshot
+
+We provide online demo, demo video and screenshots to get you started quickly.
+
+[Demo](https://demo.jumpserver.org/auth/login/?next=/)
+[Video](https://fit2cloud2-offline-installer.oss-cn-beijing.aliyuncs.com/tools/Jumpserver%20%E4%BB%8B%E7%BB%8Dv1.4.mp4)
+[Snapshot](http://docs.jumpserver.org/zh/docs/snapshot.html)
+
+### SDK
+
+We provide the SDK for your other systems to quickly interact with the Jumpserver API.
+
+- [Python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver other components use this SDK to complete the interaction.
+- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) 恺珺同学提供的Java版本的SDK thanks to 恺珺 for provide Java SDK
+
+
+### License & Copyright
+Copyright (c) 2014-2019 Beijing Duizhan Tech, Inc., All rights reserved.
+
+Licensed under The GNU General Public License version 2 (GPLv2)  (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
+
+https://www.gnu.org/licenses/gpl-2.0.html
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

apps/__init__.py

@@ -1,5 +1,3 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
-# 
-
-__version__ = "0.5.0"
+#

apps/applications/admin.py

@@ -0,0 +1,3 @@
+from django.contrib import admin
+
+# Register your models here.

apps/applications/api/__init__.py

@@ -0,0 +1 @@
+from .remote_app import *

apps/applications/api/remote_app.py

@@ -0,0 +1,31 @@
+# coding: utf-8
+#
+
+
+from rest_framework import generics
+from rest_framework.pagination import LimitOffsetPagination
+from rest_framework_bulk import BulkModelViewSet
+
+from ..hands import IsOrgAdmin, IsAppUser
+from ..models import RemoteApp
+from ..serializers import RemoteAppSerializer, RemoteAppConnectionInfoSerializer
+
+
+__all__ = [
+    'RemoteAppViewSet', 'RemoteAppConnectionInfoApi',
+]
+
+
+class RemoteAppViewSet(BulkModelViewSet):
+    filter_fields = ('name',)
+    search_fields = filter_fields
+    permission_classes = (IsOrgAdmin,)
+    queryset = RemoteApp.objects.all()
+    serializer_class = RemoteAppSerializer
+    pagination_class = LimitOffsetPagination
+
+
+class RemoteAppConnectionInfoApi(generics.RetrieveAPIView):
+    queryset = RemoteApp.objects.all()
+    permission_classes = (IsAppUser, )
+    serializer_class = RemoteAppConnectionInfoSerializer

apps/applications/apps.py

@@ -0,0 +1,7 @@
+from __future__ import unicode_literals
+
+from django.apps import AppConfig
+
+
+class ApplicationsConfig(AppConfig):
+    name = 'applications'

apps/applications/const.py

@@ -0,0 +1,68 @@
+#  coding: utf-8
+#
+
+from django.utils.translation import ugettext_lazy as _
+
+
+# RemoteApp
+REMOTE_APP_BOOT_PROGRAM_NAME = '||jmservisor'
+
+REMOTE_APP_TYPE_CHROME = 'chrome'
+REMOTE_APP_TYPE_MYSQL_WORKBENCH = 'mysql_workbench'
+REMOTE_APP_TYPE_VMWARE_CLIENT = 'vmware_client'
+REMOTE_APP_TYPE_CUSTOM = 'custom'
+
+REMOTE_APP_TYPE_CHOICES = (
+    (
+        _('Browser'),
+        (
+            (REMOTE_APP_TYPE_CHROME, 'Chrome'),
+        )
+    ),
+    (
+        _('Database tools'),
+        (
+            (REMOTE_APP_TYPE_MYSQL_WORKBENCH, 'MySQL Workbench'),
+        )
+    ),
+    (
+        _('Virtualization tools'),
+        (
+            (REMOTE_APP_TYPE_VMWARE_CLIENT, 'vSphere Client'),
+        )
+    ),
+    (REMOTE_APP_TYPE_CUSTOM, _('Custom')),
+
+)
+
+# Fields attribute write_only default => False
+
+REMOTE_APP_TYPE_CHROME_FIELDS = [
+    {'name': 'chrome_target'},
+    {'name': 'chrome_username'},
+    {'name': 'chrome_password', 'write_only': True}
+]
+REMOTE_APP_TYPE_MYSQL_WORKBENCH_FIELDS = [
+    {'name': 'mysql_workbench_ip'},
+    {'name': 'mysql_workbench_name'},
+    {'name': 'mysql_workbench_username'},
+    {'name': 'mysql_workbench_password', 'write_only': True}
+]
+REMOTE_APP_TYPE_VMWARE_CLIENT_FIELDS = [
+    {'name': 'vmware_target'},
+    {'name': 'vmware_username'},
+    {'name': 'vmware_password', 'write_only': True}
+]
+REMOTE_APP_TYPE_CUSTOM_FIELDS = [
+    {'name': 'custom_cmdline'},
+    {'name': 'custom_target'},
+    {'name': 'custom_username'},
+    {'name': 'custom_password', 'write_only': True}
+]
+
+REMOTE_APP_TYPE_MAP_FIELDS = {
+    REMOTE_APP_TYPE_CHROME: REMOTE_APP_TYPE_CHROME_FIELDS,
+    REMOTE_APP_TYPE_MYSQL_WORKBENCH: REMOTE_APP_TYPE_MYSQL_WORKBENCH_FIELDS,
+    REMOTE_APP_TYPE_VMWARE_CLIENT: REMOTE_APP_TYPE_VMWARE_CLIENT_FIELDS,
+    REMOTE_APP_TYPE_CUSTOM: REMOTE_APP_TYPE_CUSTOM_FIELDS
+}

apps/applications/forms/__init__.py

@@ -0,0 +1 @@
+from .remote_app import *

apps/applications/forms/remote_app.py

@@ -0,0 +1,130 @@
+#  coding: utf-8
+#
+
+from django.utils.translation import ugettext as _
+from django import forms
+
+from orgs.mixins import OrgModelForm
+from assets.models import SystemUser
+
+from ..models import RemoteApp
+from .. import const
+
+
+__all__ = [
+    'RemoteAppCreateUpdateForm',
+]
+
+
+class RemoteAppTypeChromeForm(forms.ModelForm):
+    chrome_target = forms.CharField(
+        max_length=128, label=_('Target URL'), required=False
+    )
+    chrome_username = forms.CharField(
+        max_length=128, label=_('Login username'), required=False
+    )
+    chrome_password = forms.CharField(
+        widget=forms.PasswordInput, strip=True,
+        max_length=128, label=_('Login password'), required=False
+    )
+
+
+class RemoteAppTypeMySQLWorkbenchForm(forms.ModelForm):
+    mysql_workbench_ip = forms.CharField(
+        max_length=128, label=_('Database IP'), required=False
+    )
+    mysql_workbench_name = forms.CharField(
+        max_length=128, label=_('Database name'), required=False
+    )
+    mysql_workbench_username = forms.CharField(
+        max_length=128, label=_('Database username'), required=False
+    )
+    mysql_workbench_password = forms.CharField(
+        widget=forms.PasswordInput, strip=True,
+        max_length=128, label=_('Database password'), required=False
+    )
+
+
+class RemoteAppTypeVMwareForm(forms.ModelForm):
+    vmware_target = forms.CharField(
+        max_length=128, label=_('Target address'), required=False
+    )
+    vmware_username = forms.CharField(
+        max_length=128, label=_('Login username'), required=False
+    )
+    vmware_password = forms.CharField(
+        widget=forms.PasswordInput, strip=True,
+        max_length=128, label=_('Login password'), required=False
+    )
+
+
+class RemoteAppTypeCustomForm(forms.ModelForm):
+    custom_cmdline = forms.CharField(
+        max_length=128, label=_('Operating parameter'), required=False
+    )
+    custom_target = forms.CharField(
+        max_length=128, label=_('Target address'), required=False
+    )
+    custom_username = forms.CharField(
+        max_length=128, label=_('Login username'), required=False
+    )
+    custom_password = forms.CharField(
+        widget=forms.PasswordInput, strip=True,
+        max_length=128, label=_('Login password'), required=False
+    )
+
+
+class RemoteAppTypeForms(
+    RemoteAppTypeChromeForm,
+    RemoteAppTypeMySQLWorkbenchForm,
+    RemoteAppTypeVMwareForm,
+    RemoteAppTypeCustomForm
+):
+    pass
+
+
+class RemoteAppCreateUpdateForm(RemoteAppTypeForms, OrgModelForm):
+    def __init__(self, *args, **kwargs):
+        # 过滤RDP资产和系统用户
+        super().__init__(*args, **kwargs)
+        field_asset = self.fields['asset']
+        field_asset.queryset = field_asset.queryset.has_protocol('rdp')
+        field_system_user = self.fields['system_user']
+        field_system_user.queryset = field_system_user.queryset.filter(
+            protocol=SystemUser.PROTOCOL_RDP
+        )
+
+    class Meta:
+        model = RemoteApp
+        fields = [
+            'name', 'asset', 'system_user', 'type', 'path', 'comment'
+        ]
+        widgets = {
+            'asset': forms.Select(attrs={
+                'class': 'select2', 'data-placeholder': _('Asset')
+            }),
+            'system_user': forms.Select(attrs={
+                'class': 'select2', 'data-placeholder': _('System user')
+            })
+        }
+
+    def _clean_params(self):
+        app_type = self.data.get('type')
+        fields = const.REMOTE_APP_TYPE_MAP_FIELDS.get(app_type, [])
+        params = {}
+        for field in fields:
+            name = field['name']
+            value = self.cleaned_data[name]
+            params.update({name: value})
+        return params
+
+    def _save_params(self, instance):
+        params = self._clean_params()
+        instance.params = params
+        instance.save()
+        return instance
+
+    def save(self, commit=True):
+        instance = super().save(commit=commit)
+        instance = self._save_params(instance)
+        return instance

apps/applications/hands.py

@@ -0,0 +1,15 @@
+"""
+    jumpserver.__app__.hands.py
+    ~~~~~~~~~~~~~~~~~
+
+    This app depends other apps api, function .. should be import or write mack here.
+
+    Other module of this app shouldn't connect with other app.
+
+    :copyright: (c) 2014-2018 by Jumpserver Team.
+    :license: GPL v2, see LICENSE for more details.
+"""
+
+
+from common.permissions import IsAppUser, IsOrgAdmin, IsValidUser, IsOrgAdminOrAppUser
+from users.models import User, UserGroup

apps/applications/migrations/0001_initial.py

@@ -0,0 +1,42 @@
+# Generated by Django 2.1.7 on 2019-05-20 11:04
+
+import common.fields.model
+from django.db import migrations, models
+import django.db.models.deletion
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+        ('assets', '0026_auto_20190325_2035'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='RemoteApp',
+            fields=[
+                ('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(max_length=128, verbose_name='Name')),
+                ('type', models.CharField(choices=[('Browser', (('chrome', 'Chrome'),)), ('Database tools', (('mysql_workbench', 'MySQL Workbench'),)), ('Virtualization tools', (('vmware_client', 'vSphere Client'),)), ('custom', 'Custom')], default='chrome', max_length=128, verbose_name='App type')),
+                ('path', models.CharField(max_length=128, verbose_name='App path')),
+                ('params', common.fields.model.EncryptJsonDictTextField(blank=True, default={}, max_length=4096, null=True, verbose_name='Parameters')),
+                ('created_by', models.CharField(blank=True, max_length=32, null=True, verbose_name='Created by')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+                ('comment', models.TextField(blank=True, default='', max_length=128, verbose_name='Comment')),
+                ('asset', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='assets.Asset', verbose_name='Asset')),
+                ('system_user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='assets.SystemUser', verbose_name='System user')),
+            ],
+            options={
+                'verbose_name': 'RemoteApp',
+                'ordering': ('name',),
+            },
+        ),
+        migrations.AlterUniqueTogether(
+            name='remoteapp',
+            unique_together={('org_id', 'name')},
+        ),
+    ]

apps/applications/models/__init__.py

@@ -0,0 +1 @@
+from .remote_app import *

apps/applications/models/remote_app.py

@@ -0,0 +1,89 @@
+# coding: utf-8
+#
+
+import uuid
+from django.db import models
+from django.utils.translation import ugettext_lazy as _
+
+from orgs.mixins import OrgModelMixin
+from common.fields.model import EncryptJsonDictTextField
+
+from .. import const
+
+
+__all__ = [
+    'RemoteApp',
+]
+
+
+class RemoteApp(OrgModelMixin):
+    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
+    name = models.CharField(max_length=128, verbose_name=_('Name'))
+    asset = models.ForeignKey(
+        'assets.Asset', on_delete=models.CASCADE, verbose_name=_('Asset')
+    )
+    system_user = models.ForeignKey(
+        'assets.SystemUser', on_delete=models.CASCADE,
+        verbose_name=_('System user')
+    )
+    type = models.CharField(
+        default=const.REMOTE_APP_TYPE_CHROME,
+        choices=const.REMOTE_APP_TYPE_CHOICES,
+        max_length=128, verbose_name=_('App type')
+    )
+    path = models.CharField(
+        max_length=128, blank=False, null=False,
+        verbose_name=_('App path')
+    )
+    params = EncryptJsonDictTextField(
+        max_length=4096, default={}, blank=True, null=True,
+        verbose_name=_('Parameters')
+    )
+    created_by = models.CharField(
+        max_length=32, null=True, blank=True, verbose_name=_('Created by')
+    )
+    date_created = models.DateTimeField(
+        auto_now_add=True, null=True, blank=True, verbose_name=_('Date created')
+    )
+    comment = models.TextField(
+        max_length=128, default='', blank=True, verbose_name=_('Comment')
+    )
+
+    class Meta:
+        verbose_name = _("RemoteApp")
+        unique_together = [('org_id', 'name')]
+        ordering = ('name', )
+
+    def __str__(self):
+        return self.name
+
+    @property
+    def parameters(self):
+        """
+        返回Guacamole需要的RemoteApp配置参数信息中的parameters参数
+        """
+        _parameters = list()
+        _parameters.append(self.type)
+        path = '\"%s\"' % self.path
+        _parameters.append(path)
+        for field in const.REMOTE_APP_TYPE_MAP_FIELDS[self.type]:
+            value = self.params.get(field['name'])
+            if value is None:
+                continue
+            _parameters.append(value)
+        _parameters = ' '.join(_parameters)
+        return _parameters
+
+    @property
+    def asset_info(self):
+        return {
+            'id': self.asset.id,
+            'hostname': self.asset.hostname
+        }
+
+    @property
+    def system_user_info(self):
+        return {
+            'id': self.system_user.id,
+            'name': self.system_user.name
+        }

apps/applications/serializers/__init__.py

@@ -0,0 +1 @@
+from .remote_app import *

apps/applications/serializers/remote_app.py

@@ -0,0 +1,103 @@
+# coding: utf-8
+#
+
+
+from rest_framework import serializers
+
+from common.serializers import AdaptedBulkListSerializer
+from orgs.mixins import BulkOrgResourceModelSerializer
+
+from .. import const
+from ..models import RemoteApp
+
+
+__all__ = [
+    'RemoteAppSerializer', 'RemoteAppConnectionInfoSerializer',
+]
+
+
+class RemoteAppParamsDictField(serializers.DictField):
+    """
+    RemoteApp field => params
+    """
+    @staticmethod
+    def filter_attribute(attribute, instance):
+        """
+        过滤掉params字段值中write_only特性的key-value值
+        For example, the chrome_password field is not returned when serializing
+        {
+            'chrome_target': 'http://www.jumpserver.org/',
+            'chrome_username': 'admin',
+            'chrome_password': 'admin',
+        }
+        """
+        for field in const.REMOTE_APP_TYPE_MAP_FIELDS[instance.type]:
+            if field.get('write_only', False):
+                attribute.pop(field['name'], None)
+        return attribute
+
+    def get_attribute(self, instance):
+        """
+        序列化时调用
+        """
+        attribute = super().get_attribute(instance)
+        attribute = self.filter_attribute(attribute, instance)
+        return attribute
+
+    @staticmethod
+    def filter_value(dictionary, value):
+        """
+        过滤掉不属于当前app_type所包含的key-value值
+        """
+        app_type = dictionary.get('type', const.REMOTE_APP_TYPE_CHROME)
+        fields = const.REMOTE_APP_TYPE_MAP_FIELDS[app_type]
+        fields_names = [field['name'] for field in fields]
+        no_need_keys = [k for k in value.keys() if k not in fields_names]
+        for k in no_need_keys:
+            value.pop(k)
+        return value
+
+    def get_value(self, dictionary):
+        """
+        反序列化时调用
+        """
+        value = super().get_value(dictionary)
+        value = self.filter_value(dictionary, value)
+        return value
+
+
+class RemoteAppSerializer(BulkOrgResourceModelSerializer):
+    params = RemoteAppParamsDictField()
+
+    class Meta:
+        model = RemoteApp
+        list_serializer_class = AdaptedBulkListSerializer
+        fields = [
+            'id', 'name', 'asset', 'system_user', 'type', 'path', 'params',
+            'comment', 'created_by', 'date_created', 'asset_info',
+            'system_user_info', 'get_type_display',
+        ]
+        read_only_fields = [
+            'created_by', 'date_created', 'asset_info',
+            'system_user_info', 'get_type_display'
+        ]
+
+
+class RemoteAppConnectionInfoSerializer(serializers.ModelSerializer):
+    parameter_remote_app = serializers.SerializerMethodField()
+
+    class Meta:
+        model = RemoteApp
+        fields = [
+            'id', 'name', 'asset', 'system_user', 'parameter_remote_app',
+        ]
+        read_only_fields = ['parameter_remote_app']
+
+    @staticmethod
+    def get_parameter_remote_app(obj):
+        parameter = {
+            'program': const.REMOTE_APP_BOOT_PROGRAM_NAME,
+            'working_directory': '',
+            'parameters': obj.parameters,
+        }
+        return parameter

apps/applications/templates/applications/remote_app_create_update.html

@@ -0,0 +1,123 @@
+{% extends '_base_create_update.html' %}
+{% load static %}
+{% load bootstrap3 %}
+{% load i18n %}
+
+{% block form %}
+    <form id="appForm" method="post" class="form-horizontal">
+        {% if form.non_field_errors %}
+            <div class="alert alert-danger">
+                {{ form.non_field_errors }}
+            </div>
+        {% endif %}
+        {% csrf_token %}
+        {% bootstrap_field form.name layout="horizontal" %}
+        {% bootstrap_field form.asset layout="horizontal" %}
+        {% bootstrap_field form.system_user layout="horizontal" %}
+        {% bootstrap_field form.type layout="horizontal" %}
+        {% bootstrap_field form.path layout="horizontal" %}
+
+        <div class="hr-line-dashed"></div>
+
+        {# chrome #}
+        <div class="chrome-fields">
+        {% bootstrap_field form.chrome_target layout="horizontal" %}
+        {% bootstrap_field form.chrome_username layout="horizontal" %}
+        {% bootstrap_field form.chrome_password layout="horizontal" %}
+        </div>
+
+        {# mysql workbench #}
+        <div class="mysql_workbench-fields">
+            {% bootstrap_field form.mysql_workbench_ip layout="horizontal" %}
+            {% bootstrap_field form.mysql_workbench_name layout="horizontal" %}
+            {% bootstrap_field form.mysql_workbench_username layout="horizontal" %}
+            {% bootstrap_field form.mysql_workbench_password layout="horizontal" %}
+        </div>
+
+        {# vmware #}
+        <div class="vmware_client-fields">
+        {% bootstrap_field form.vmware_target layout="horizontal" %}
+        {% bootstrap_field form.vmware_username layout="horizontal" %}
+        {% bootstrap_field form.vmware_password layout="horizontal" %}
+        </div>
+
+        {# custom #}
+        <div class="custom-fields">
+        {% bootstrap_field form.custom_cmdline layout="horizontal" %}
+        {% bootstrap_field form.custom_target layout="horizontal" %}
+        {% bootstrap_field form.custom_username layout="horizontal" %}
+        {% bootstrap_field form.custom_password layout="horizontal" %}
+        </div>
+
+        {% bootstrap_field form.comment layout="horizontal" %}
+        <div class="hr-line-dashed"></div>
+        <div class="form-group">
+            <div class="col-sm-4 col-sm-offset-2">
+                <button class="btn btn-default" type="reset"> {% trans 'Reset' %}</button>
+
+                <button id="submit_button" class="btn btn-primary" type="submit">{% trans 'Submit' %}</button>
+            </div>
+        </div>
+
+    </form>
+{% endblock %}
+
+{% block custom_foot_js %}
+<script type="text/javascript">
+var app_type_id = '#' + '{{ form.type.id_for_label }}';
+var app_path_id = '#' + '{{ form.path.id_for_label }}';
+var all_type_fields = [
+    '.chrome-fields',
+    '.mysql_workbench-fields',
+    '.vmware_client-fields',
+    '.custom-fields'
+];
+var app_type_map_default_fields_value = {
+    'chrome': {
+        'app_path': 'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe'
+    },
+    'mysql_workbench': {
+        'app_path': 'C:\\Program Files\\MySQL\\MySQL Workbench 8.0 CE\\MySQLWorkbench.exe'
+    },
+    'vmware_client': {
+        'app_path': 'C:\\Program Files (x86)\\VMware\\Infrastructure\\Virtual Infrastructure Client\\Launcher\\VpxClient.exe'
+    },
+    'custom': {'app_path': ''}
+};
+function getAppType(){
+    return $(app_type_id+ " option:selected").val();
+}
+function initialDefaultValue(){
+    var app_type = getAppType();
+    var app_path = $(app_path_id).val();
+    if(app_path){
+        app_type_map_default_fields_value[app_type]['app_path'] = app_path
+    }
+}
+function setDefaultValue(){
+    // 设置类型相关字段的默认值
+    var app_type = getAppType();
+    var app_path = app_type_map_default_fields_value[app_type]['app_path'];
+    $(app_path_id).val(app_path)
+}
+function hiddenFields(){
+    var app_type = getAppType();
+    $.each(all_type_fields, function(index, value){
+        $(value).addClass('hidden')
+    });
+    $('.' + app_type + '-fields').removeClass('hidden');
+}
+$(document).ready(function () {
+    $('.select2').select2({
+        closeOnSelect: true
+    });
+    initialDefaultValue();
+    hiddenFields();
+    setDefaultValue();
+})
+.on('change', app_type_id, function(){
+    hiddenFields();
+    setDefaultValue();
+});
+</script>
+{% endblock %}

apps/applications/templates/applications/remote_app_detail.html

@@ -0,0 +1,109 @@
+{% extends 'base.html' %}
+{% load static %}
+{% load i18n %}
+
+{% block custom_head_css_js %}
+    <link href="{% static 'css/plugins/select2/select2.min.css' %}" rel="stylesheet">
+    <script src="{% static 'js/plugins/select2/select2.full.min.js' %}"></script>
+{% endblock %}
+
+{% block content %}
+    <div class="wrapper wrapper-content animated fadeInRight">
+        <div class="row">
+            <div class="col-sm-12">
+                <div class="ibox float-e-margins">
+                    <div class="panel-options">
+                        <ul class="nav nav-tabs">
+                            <li class="active">
+                                <a href="{% url 'applications:remote-app-detail' pk=remote_app.id %}" class="text-center"><i class="fa fa-laptop"></i> {% trans 'Detail' %} </a>
+                            </li>
+                            <li class="pull-right">
+                                <a class="btn btn-outline btn-default" href="{% url 'applications:remote-app-update' pk=remote_app.id %}"><i class="fa fa-edit"></i>{% trans 'Update' %}</a>
+                            </li>
+                            <li class="pull-right">
+                                <a class="btn btn-outline btn-danger btn-delete-application">
+                                    <i class="fa fa-trash-o"></i>{% trans 'Delete' %}
+                                </a>
+                            </li>
+                        </ul>
+                    </div>
+                    <div class="tab-content">
+                        <div class="col-sm-8" style="padding-left: 0;">
+                            <div class="ibox float-e-margins">
+                                <div class="ibox-title">
+                                    <span class="label"><b>{{ remote_app.name }}</b></span>
+                                    <div class="ibox-tools">
+                                        <a class="collapse-link">
+                                            <i class="fa fa-chevron-up"></i>
+                                        </a>
+                                        <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+                                            <i class="fa fa-wrench"></i>
+                                        </a>
+                                        <ul class="dropdown-menu dropdown-user">
+                                        </ul>
+                                        <a class="close-link">
+                                            <i class="fa fa-times"></i>
+                                        </a>
+                                    </div>
+                                </div>
+                                <div class="ibox-content">
+                                    <table class="table">
+                                        <tbody>
+                                        <tr class="no-borders-tr">
+                                            <td>{% trans 'Name' %}:</td>
+                                            <td><b>{{ remote_app.name }}</b></td>
+                                        </tr>
+                                        <tr>
+                                            <td>{% trans 'Asset' %}:</td>
+                                            <td><b><a href="{% url 'assets:asset-detail' pk=remote_app.asset.id %}">{{ remote_app.asset.hostname }}</a></b></td>
+                                        </tr>
+                                        <tr>
+                                            <td>{% trans 'System user' %}:</td>
+                                            <td><b><a href="{% url 'assets:system-user-detail' pk=remote_app.system_user.id %}">{{ remote_app.system_user.name }}</a></b></td>
+                                        </tr>
+                                        <tr>
+                                            <td>{% trans 'App type' %}:</td>
+                                            <td><b>{{ remote_app.get_type_display }}</b></td>
+                                        </tr>
+                                        <tr>
+                                            <td>{% trans 'App path' %}:</td>
+                                            <td><b>{{ remote_app.path }}</b></td>
+                                        </tr>
+                                        <tr>
+                                            <td>{% trans 'Date created' %}:</td>
+                                            <td><b>{{ remote_app.date_created }}</b></td>
+                                        </tr>
+                                        <tr>
+                                            <td>{% trans 'Created by' %}:</td>
+                                            <td><b>{{ remote_app.created_by  }}</b></td>
+                                        </tr>
+                                        <tr>
+                                            <td>{% trans 'Comment' %}:</td>
+                                            <td><b>{{ remote_app.comment }}</b></td>
+                                        </tr>
+                                        </tbody>
+                                    </table>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+{% endblock %}
+{% block custom_foot_js %}
+<script>
+jumpserver.nodes_selected = {};
+$(document).ready(function () {
+})
+.on('click', '.btn-delete-application', function () {
+    var $this = $(this);
+    var name = "{{ remote_app.name }}";
+    var rid = "{{ remote_app.id }}";
+    var the_url = '{% url "api-applications:remote-app-detail" pk=DEFAULT_PK %}'.replace('{{ DEFAULT_PK }}', rid);
+    var redirect_url = "{% url 'applications:remote-app-list' %}";
+    objectDelete($this, name, the_url, redirect_url);
+})
+</script>
+{% endblock %}

apps/applications/templates/applications/remote_app_list.html

@@ -0,0 +1,90 @@
+{% extends '_base_list.html' %}
+{% load i18n static %}
+{% block help_message %}
+<div class="alert alert-info help-message">
+    {% trans 'Before using this feature, make sure that the application loader has been uploaded to the application server and successfully published as a RemoteApp application' %}
+    <b><a href="https://github.com/jumpserver/Jmservisor/releases" target="view_window" >{% trans 'Download application loader' %}</a></b>
+</div>
+{% endblock %}
+{% block table_search %}{% endblock %}
+{% block table_container %}
+    <div class="uc pull-left  m-r-5">
+        <a href="{% url 'applications:remote-app-create' %}" class="btn btn-sm btn-primary"> {% trans "Create RemoteApp" %} </a>
+    </div>
+    <table class="table table-striped table-bordered table-hover " id="remote_app_list_table" >
+        <thead>
+        <tr>
+            <th class="text-center">
+                <input type="checkbox" id="check_all" class="ipt_check_all" >
+            </th>
+            <th class="text-center">{% trans 'Name' %}</th>
+            <th class="text-center">{% trans 'App type' %}</th>
+            <th class="text-center">{% trans 'Asset' %}</th>
+            <th class="text-center">{% trans 'System user' %}</th>
+            <th class="text-center">{% trans 'Comment' %}</th>
+            <th class="text-center">{% trans 'Action' %}</th>
+        </tr>
+        </thead>
+        <tbody>
+        </tbody>
+    </table>
+{% endblock %}
+{% block content_bottom_left %}{% endblock %}
+{% block custom_foot_js %}
+<script>
+function initTable() {
+    var options = {
+        ele: $('#remote_app_list_table'),
+        columnDefs: [
+            {targets: 1, createdCell: function (td, cellData, rowData) {
+                    cellData = htmlEscape(cellData);
+                    {% url 'applications:remote-app-detail' pk=DEFAULT_PK as the_url  %}
+                    var detail_btn = '<a href="{{ the_url }}">' + cellData + '</a>';
+                    $(td).html(detail_btn.replace('{{ DEFAULT_PK }}', rowData.id));
+                }},
+            {targets: 3, createdCell: function (td, cellData, rowData) {
+                    var hostname = htmlEscape(cellData.hostname);
+                    var detail_btn = '<a href="{% url 'assets:asset-detail' pk=DEFAULT_PK %}">' + hostname + '</a>';
+                    $(td).html(detail_btn.replace('{{ DEFAULT_PK }}', cellData.id));
+                }},
+            {targets: 4, createdCell: function (td, cellData, rowData) {
+                    var name = htmlEscape(cellData.name);
+                    var detail_btn = '<a href="{% url 'assets:system-user-detail' pk=DEFAULT_PK %}">' + name + '</a>';
+                    $(td).html(detail_btn.replace('{{ DEFAULT_PK }}', cellData.id));
+                }},
+            {targets: 6, createdCell: function (td, cellData, rowData) {
+                    var update_btn = '<a href="{% url "applications:remote-app-update" pk=DEFAULT_PK %}" class="btn btn-xs btn-info">{% trans "Update" %}</a>'.replace("{{ DEFAULT_PK }}", cellData);
+                    var del_btn = '<a class="btn btn-xs btn-danger m-l-xs btn-delete" data-rid="{{ DEFAULT_PK }}">{% trans "Delete" %}</a>'.replace('{{ DEFAULT_PK }}', cellData);
+                    $(td).html(update_btn + del_btn)
+                }}
+        ],
+        ajax_url: '{% url "api-applications:remote-app-list" %}',
+        columns: [
+            {data: "id"},
+            {data: "name" },
+            {data: "get_type_display", orderable: false},
+            {data: "asset_info", orderable: false},
+            {data: "system_user_info", orderable: false},
+            {data: "comment"},
+            {data: "id", orderable: false}
+        ],
+        op_html: $('#actions').html()
+    };
+    jumpserver.initServerSideDataTable(options);
+}
+$(document).ready(function(){
+    initTable();
+})
+.on('click', '.btn-delete', function () {
+    var $this = $(this);
+    var $data_table = $('#remote_app_list_table').DataTable();
+    var name = $(this).closest("tr").find(":nth-child(2)").children('a').html();
+    var rid = $this.data('rid');
+    var the_url = '{% url "api-applications:remote-app-detail" pk=DEFAULT_PK %}'.replace('{{ DEFAULT_PK }}', rid);
+    objectDelete($this, name, the_url);
+    setTimeout( function () {
+        $data_table.ajax.reload();
+    }, 3000);
+});
+</script>
+{% endblock %}

apps/applications/templates/applications/user_remote_app_list.html

@@ -0,0 +1,79 @@
+{% extends 'base.html' %}
+{% load i18n static %}
+
+{% block custom_head_css_js %}
+    <script src="{% static 'js/jquery.form.min.js' %}"></script>
+{% endblock %}
+
+{% block content %}
+<div class="mail-box-header">
+    <table class="table table-striped table-bordered table-hover " id="remote_app_list_table" >
+        <thead>
+        <tr>
+            <th class="text-center">
+                <input type="checkbox" id="check_all" class="ipt_check_all" >
+            </th>
+            <th class="text-center">{% trans 'Name' %}</th>
+            <th class="text-center">{% trans 'App type' %}</th>
+            <th class="text-center">{% trans 'Asset' %}</th>
+            <th class="text-center">{% trans 'System user' %}</th>
+            <th class="text-center">{% trans 'Comment' %}</th>
+            <th class="text-center">{% trans 'Action' %}</th>
+        </tr>
+        </thead>
+        <tbody>
+        </tbody>
+    </table>
+</div>
+{% endblock %}
+{% block custom_foot_js %}
+<script>
+var inited = false;
+var remote_app_table, url;
+
+function initTable() {
+    if (inited){
+        return
+    } else {
+        inited = true;
+    }
+    url = '{% url "api-perms:my-remote-apps" %}';
+    var options = {
+        ele: $('#remote_app_list_table'),
+        columnDefs: [
+            {targets: 1, createdCell: function (td, cellData, rowData) {
+                    var name = htmlEscape(cellData);
+                    $(td).html(name)
+                }},
+            {targets: 3, createdCell: function (td, cellData, rowData) {
+                    var hostname = htmlEscape(cellData.hostname);
+                    $(td).html(hostname);
+                }},
+            {targets: 4, createdCell: function (td, cellData, rowData) {
+                    var name = htmlEscape(cellData.name);
+                    $(td).html(name);
+                }},
+            {targets: 6, createdCell: function (td, cellData, rowData) {
+                    var conn_btn = '<a href="{% url "luna-view" %}?login_to=' +  cellData +'" class="btn btn-xs btn-primary">{% trans "Connect" %}</a>'.replace("{{ DEFAULT_PK }}", cellData);
+                    $(td).html(conn_btn)
+                }}
+        ],
+        ajax_url: url,
+        columns: [
+            {data: "id"},
+            {data: "name"},
+            {data: "get_type_display", orderable: false},
+            {data: "asset_info", orderable: false},
+            {data: "system_user_info", orderable: false},
+            {data: "comment", orderable: false},
+            {data: "id", orderable: false}
+        ]
+    };
+    remote_app_table = jumpserver.initServerSideDataTable(options);
+    return remote_app_table
+}
+$(document).ready(function(){
+    initTable();
+})
+</script>
+{% endblock %}

apps/applications/tests.py

@@ -0,0 +1,3 @@
+from django.test import TestCase
+
+# Create your tests here.

apps/applications/urls/__init__.py

@@ -0,0 +1,7 @@
+#  coding: utf-8
+#
+
+
+__all__ = [
+
+]

apps/applications/urls/api_urls.py

@@ -0,0 +1,20 @@
+# coding:utf-8
+#
+
+from django.urls import path
+from rest_framework_bulk.routes import BulkRouter
+
+from .. import api
+
+app_name = 'applications'
+
+router = BulkRouter()
+router.register(r'remote-app', api.RemoteAppViewSet, 'remote-app')
+
+urlpatterns = [
+    path('remote-apps/<uuid:pk>/connection-info/',
+         api.RemoteAppConnectionInfoApi.as_view(),
+         name='remote-app-connection-info')
+]
+
+urlpatterns += router.urls

apps/applications/urls/views_urls.py

@@ -0,0 +1,16 @@
+# coding:utf-8
+from django.urls import path
+from .. import views
+
+app_name = 'applications'
+
+urlpatterns = [
+    # RemoteApp
+    path('remote-app/', views.RemoteAppListView.as_view(), name='remote-app-list'),
+    path('remote-app/create/', views.RemoteAppCreateView.as_view(), name='remote-app-create'),
+    path('remote-app/<uuid:pk>/update/', views.RemoteAppUpdateView.as_view(), name='remote-app-update'),
+    path('remote-app/<uuid:pk>/', views.RemoteAppDetailView.as_view(), name='remote-app-detail'),
+    # User RemoteApp view
+    path('user-remote-app/', views.UserRemoteAppListView.as_view(), name='user-remote-app-list')
+
+]

apps/applications/views/__init__.py

@@ -0,0 +1 @@
+from .remote_app import *

apps/applications/views/remote_app.py

@@ -0,0 +1,103 @@
+#  coding: utf-8
+#
+
+from django.utils.translation import ugettext as _
+from django.views.generic import TemplateView
+from django.views.generic.edit import CreateView, UpdateView
+from django.views.generic.detail import DetailView
+from django.contrib.messages.views import SuccessMessageMixin
+from django.urls import reverse_lazy
+
+
+from common.permissions import PermissionsMixin, IsOrgAdmin, IsValidUser
+from common.const import create_success_msg, update_success_msg
+
+from ..models import RemoteApp
+from .. import forms
+
+
+__all__ = [
+    'RemoteAppListView', 'RemoteAppCreateView', 'RemoteAppUpdateView',
+    'RemoteAppDetailView', 'UserRemoteAppListView',
+]
+
+
+class RemoteAppListView(PermissionsMixin, TemplateView):
+    template_name = 'applications/remote_app_list.html'
+    permission_classes = [IsOrgAdmin]
+
+    def get_context_data(self, **kwargs):
+        context = {
+            'app': _('Applications'),
+            'action': _('RemoteApp list'),
+        }
+        kwargs.update(context)
+        return super().get_context_data(**kwargs)
+
+
+class RemoteAppCreateView(PermissionsMixin, SuccessMessageMixin, CreateView):
+    template_name = 'applications/remote_app_create_update.html'
+    model = RemoteApp
+    form_class = forms.RemoteAppCreateUpdateForm
+    success_url = reverse_lazy('applications:remote-app-list')
+    permission_classes = [IsOrgAdmin]
+
+    def get_context_data(self, **kwargs):
+        context = {
+            'app': _('Applications'),
+            'action': _('Create RemoteApp'),
+        }
+        kwargs.update(context)
+        return super().get_context_data(**kwargs)
+
+    def get_success_message(self, cleaned_data):
+        return create_success_msg % ({'name': cleaned_data['name']})
+
+
+class RemoteAppUpdateView(PermissionsMixin, SuccessMessageMixin, UpdateView):
+    template_name = 'applications/remote_app_create_update.html'
+    model = RemoteApp
+    form_class = forms.RemoteAppCreateUpdateForm
+    success_url = reverse_lazy('applications:remote-app-list')
+    permission_classes = [IsOrgAdmin]
+
+    def get_initial(self):
+        return {k: v for k, v in self.object.params.items()}
+
+    def get_context_data(self, **kwargs):
+        context = {
+            'app': _('Applications'),
+            'action': _('Update RemoteApp'),
+        }
+        kwargs.update(context)
+        return super().get_context_data(**kwargs)
+
+    def get_success_message(self, cleaned_data):
+        return update_success_msg % ({'name': cleaned_data['name']})
+
+
+class RemoteAppDetailView(PermissionsMixin, DetailView):
+    template_name = 'applications/remote_app_detail.html'
+    model = RemoteApp
+    context_object_name = 'remote_app'
+    permission_classes = [IsOrgAdmin]
+
+    def get_context_data(self, **kwargs):
+        context = {
+            'app': _('Applications'),
+            'action': _('RemoteApp detail'),
+        }
+        kwargs.update(context)
+        return super().get_context_data(**kwargs)
+
+
+class UserRemoteAppListView(PermissionsMixin, TemplateView):
+    template_name = 'applications/user_remote_app_list.html'
+    permission_classes = [IsValidUser]
+
+    def get_context_data(self, **kwargs):
+        context = {
+            'action': _('My RemoteApp'),
+        }
+        kwargs.update(context)
+        return super().get_context_data(**kwargs)

apps/assets/api/__init__.py

@@ -3,3 +3,6 @@ from .asset import *
 from .label import *
 from .system_user import *
 from .node import *
+from .domain import *
+from .cmd_filter import *
+from .asset_user import *

apps/assets/api/admin_user.py

@@ -14,37 +14,55 @@
 # limitations under the License.
 
 from django.db import transaction
+from django.shortcuts import get_object_or_404
 from rest_framework import generics
 from rest_framework.response import Response
 from rest_framework_bulk import BulkModelViewSet
+from rest_framework.pagination import LimitOffsetPagination
 
-from common.mixins import IDInFilterMixin
+from common.mixins import IDInCacheFilterMixin
 from common.utils import get_logger
-from ..hands import IsSuperUser
+from ..hands import IsOrgAdmin
 from ..models import AdminUser, Asset
 from .. import serializers
-from ..tasks import test_admin_user_connectability_manual
+from ..tasks import test_admin_user_connectivity_manual
 
 
 logger = get_logger(__file__)
 __all__ = [
-    'AdminUserViewSet', 'ReplaceNodesAdminUserApi', 'AdminUserTestConnectiveApi'
+    'AdminUserViewSet', 'ReplaceNodesAdminUserApi',
+    'AdminUserTestConnectiveApi', 'AdminUserAuthApi',
+    'AdminUserAssetsListView',
 ]
 
 
-class AdminUserViewSet(IDInFilterMixin, BulkModelViewSet):
+class AdminUserViewSet(IDInCacheFilterMixin, BulkModelViewSet):
     """
     Admin user api set, for add,delete,update,list,retrieve resource
     """
+
+    filter_fields = ("name", "username")
+    search_fields = filter_fields
     queryset = AdminUser.objects.all()
     serializer_class = serializers.AdminUserSerializer
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
+    pagination_class = LimitOffsetPagination
+
+    def get_queryset(self):
+        queryset = super().get_queryset().all()
+        return queryset
+
+
+class AdminUserAuthApi(generics.UpdateAPIView):
+    queryset = AdminUser.objects.all()
+    serializer_class = serializers.AdminUserAuthSerializer
+    permission_classes = (IsOrgAdmin,)
 
 
 class ReplaceNodesAdminUserApi(generics.UpdateAPIView):
     queryset = AdminUser.objects.all()
     serializer_class = serializers.ReplaceNodeAdminUserSerializer
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
 
     def update(self, request, *args, **kwargs):
         admin_user = self.get_object()
@@ -65,12 +83,30 @@ class ReplaceNodesAdminUserApi(generics.UpdateAPIView):
 
 class AdminUserTestConnectiveApi(generics.RetrieveAPIView):
     """
-    Test asset admin user connectivity
+    Test asset admin user assets_connectivity
     """
     queryset = AdminUser.objects.all()
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.TaskIDSerializer
 
     def retrieve(self, request, *args, **kwargs):
         admin_user = self.get_object()
-        test_admin_user_connectability_manual.delay(admin_user)
-        return Response({"msg": "Task created"})
+        task = test_admin_user_connectivity_manual.delay(admin_user)
+        return Response({"task": task.id})
+
+
+class AdminUserAssetsListView(generics.ListAPIView):
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.AssetSimpleSerializer
+    pagination_class = LimitOffsetPagination
+    filter_fields = ("hostname", "ip")
+    http_method_names = ['get']
+    search_fields = filter_fields
+
+    def get_object(self):
+        pk = self.kwargs.get('pk')
+        return get_object_or_404(AdminUser, pk=pk)
+
+    def get_queryset(self):
+        admin_user = self.get_object()
+        return admin_user.get_related_assets()

apps/assets/api/asset.py

@@ -1,79 +1,129 @@
 # -*- coding: utf-8 -*-
 #
 
+import uuid
+import random
+
 from rest_framework import generics
+from rest_framework.views import APIView
 from rest_framework.response import Response
 from rest_framework_bulk import BulkModelViewSet
 from rest_framework_bulk import ListBulkCreateUpdateDestroyAPIView
 from rest_framework.pagination import LimitOffsetPagination
+from django.utils.translation import ugettext_lazy as _
 from django.shortcuts import get_object_or_404
+from django.urls import reverse_lazy
+from django.core.cache import cache
 from django.db.models import Q
 
-from common.mixins import IDInFilterMixin
-from common.utils import get_logger
-from ..hands import IsSuperUser, IsValidUser, IsSuperUserOrAppUser, \
-    NodePermissionUtil
-from ..models import  Asset, SystemUser, AdminUser, Node
+from common.mixins import IDInCacheFilterMixin, ApiMessageMixin
+
+from common.utils import get_logger, get_object_or_none
+from common.permissions import IsOrgAdmin, IsOrgAdminOrAppUser
+from orgs.mixins import OrgBulkModelViewSet
+from ..const import CACHE_KEY_ASSET_BULK_UPDATE_ID_PREFIX
+from ..models import Asset, AdminUser, Node
 from .. import serializers
 from ..tasks import update_asset_hardware_info_manual, \
-    test_asset_connectability_manual
+    test_asset_connectivity_manual
 from ..utils import LabelFilter
 
 
 logger = get_logger(__file__)
 __all__ = [
-    'AssetViewSet', 'UserAssetListView', 'AssetListUpdateApi',
-    'AssetRefreshHardwareApi', 'AssetAdminUserTestApi'
+    'AssetViewSet', 'AssetListUpdateApi',
+    'AssetRefreshHardwareApi', 'AssetAdminUserTestApi',
+    'AssetGatewayApi', 'AssetBulkUpdateSelectAPI'
 ]
 
 
-class AssetViewSet(IDInFilterMixin, LabelFilter, BulkModelViewSet):
+class AssetViewSet(LabelFilter, OrgBulkModelViewSet):
     """
     API endpoint that allows Asset to be viewed or edited.
     """
-    filter_fields = ("hostname", "ip")
-    search_fields = filter_fields
+    filter_fields = ("hostname", "ip", "systemuser__id", "admin_user__id")
+    search_fields = ("hostname", "ip")
     ordering_fields = ("hostname", "ip", "port", "cpu_cores")
     queryset = Asset.objects.all()
     serializer_class = serializers.AssetSerializer
     pagination_class = LimitOffsetPagination
-    permission_classes = (IsSuperUserOrAppUser,)
+    permission_classes = (IsOrgAdminOrAppUser,)
+    success_message = _("%(hostname)s was %(action)s successfully")
 
-    def get_queryset(self):
-        queryset = super().get_queryset()
-        admin_user_id = self.request.query_params.get('admin_user_id')
+    def set_assets_node(self, assets):
+        if not isinstance(assets, list):
+            assets = [assets]
+        node_id = self.request.query_params.get('node_id')
+        if not node_id:
+            return
+        node = get_object_or_none(Node, pk=node_id)
+        if not node:
+            return
+        node.assets.add(*assets)
+
+    def perform_create(self, serializer):
+        assets = serializer.save()
+        self.set_assets_node(assets)
+
+    def filter_node(self, queryset):
         node_id = self.request.query_params.get("node_id")
+        if not node_id:
+            return queryset
 
-        if admin_user_id:
-            admin_user = get_object_or_404(AdminUser, id=admin_user_id)
-            queryset = queryset.filter(admin_user=admin_user)
-        if node_id:
-            node = get_object_or_404(Node, id=node_id)
-            if not node.is_root():
-                queryset = queryset.filter(nodes__key__startswith=node.key).distinct()
+        node = get_object_or_404(Node, id=node_id)
+        show_current_asset = self.request.query_params.get("show_current_asset") in ('1', 'true')
+
+        if node.is_root() and show_current_asset:
+            queryset = queryset.filter(
+                Q(nodes=node_id) | Q(nodes__isnull=True)
+            )
+        elif node.is_root() and not show_current_asset:
+            pass
+        elif not node.is_root() and show_current_asset:
+            queryset = queryset.filter(nodes=node)
+        else:
+            queryset = queryset.filter(
+                nodes__key__regex='^{}(:[0-9]+)*$'.format(node.key),
+            )
+        return queryset
+
+    def filter_admin_user_id(self, queryset):
+        admin_user_id = self.request.query_params.get('admin_user_id')
+        if not admin_user_id:
+            return queryset
+        admin_user = get_object_or_404(AdminUser, id=admin_user_id)
+        queryset = queryset.filter(admin_user=admin_user)
+        return queryset
+
+    def filter_queryset(self, queryset):
+        queryset = super().filter_queryset(queryset)
+        queryset = self.filter_node(queryset)
+        queryset = self.filter_admin_user_id(queryset)
         return queryset
 
 
-class UserAssetListView(generics.ListAPIView):
-    queryset = Asset.objects.all()
-    serializer_class = serializers.AssetSerializer
-    permission_classes = (IsValidUser,)
-
-    def get_queryset(self):
-        assets_granted = NodePermissionUtil.get_user_assets(self.request.user).keys()
-        queryset = self.queryset.filter(
-            id__in=[asset.id for asset in assets_granted]
-        )
-        return queryset
-
-
-class AssetListUpdateApi(IDInFilterMixin, ListBulkCreateUpdateDestroyAPIView):
+class AssetListUpdateApi(IDInCacheFilterMixin, ListBulkCreateUpdateDestroyAPIView):
     """
     Asset bulk update api
     """
     queryset = Asset.objects.all()
     serializer_class = serializers.AssetSerializer
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
+
+
+class AssetBulkUpdateSelectAPI(APIView):
+    permission_classes = (IsOrgAdmin,)
+
+    def post(self, request, *args, **kwargs):
+        assets_id = request.data.get('assets_id', '')
+        if assets_id:
+            spm = uuid.uuid4().hex
+            key = CACHE_KEY_ASSET_BULK_UPDATE_ID_PREFIX.format(spm)
+            cache.set(key, assets_id, 300)
+            url = reverse_lazy('assets:asset-bulk-update') + '?spm=%s' % spm
+            return Response({'url': url})
+        error = _('Please select assets that need to be updated')
+        return Response({'error': error}, status=400)
 
 
 class AssetRefreshHardwareApi(generics.RetrieveAPIView):
@@ -82,31 +132,43 @@ class AssetRefreshHardwareApi(generics.RetrieveAPIView):
     """
     queryset = Asset.objects.all()
     serializer_class = serializers.AssetSerializer
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
 
     def retrieve(self, request, *args, **kwargs):
         asset_id = kwargs.get('pk')
         asset = get_object_or_404(Asset, pk=asset_id)
-        summary = update_asset_hardware_info_manual(asset)[1]
-        logger.debug("Refresh summary: {}".format(summary))
-        if summary.get('dark'):
-            return Response(summary['dark'].values(), status=501)
-        else:
-            return Response({"msg": "ok"})
+        task = update_asset_hardware_info_manual.delay(asset)
+        return Response({"task": task.id})
 
 
 class AssetAdminUserTestApi(generics.RetrieveAPIView):
     """
-    Test asset admin user connectivity
+    Test asset admin user assets_connectivity
     """
     queryset = Asset.objects.all()
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.TaskIDSerializer
 
     def retrieve(self, request, *args, **kwargs):
         asset_id = kwargs.get('pk')
         asset = get_object_or_404(Asset, pk=asset_id)
-        ok, msg = test_asset_connectability_manual(asset)
-        if ok:
-            return Response({"msg": "pong"})
+        task = test_asset_connectivity_manual.delay(asset)
+        return Response({"task": task.id})
+
+
+class AssetGatewayApi(generics.RetrieveAPIView):
+    queryset = Asset.objects.all()
+    permission_classes = (IsOrgAdminOrAppUser,)
+    serializer_class = serializers.GatewayWithAuthSerializer
+
+    def retrieve(self, request, *args, **kwargs):
+        asset_id = kwargs.get('pk')
+        asset = get_object_or_404(Asset, pk=asset_id)
+
+        if asset.domain and \
+                asset.domain.gateways.filter(protocol='ssh').exists():
+            gateway = random.choice(asset.domain.gateways.filter(protocol='ssh'))
+            serializer = serializers.GatewayWithAuthSerializer(instance=gateway)
+            return Response(serializer.data)
         else:
-            return Response({"error": msg}, status=502)
+            return Response({"msg": "Not have gateway"}, status=404)

apps/assets/api/asset_user.py

@@ -0,0 +1,180 @@
+# -*- coding: utf-8 -*-
+#
+
+from rest_framework.response import Response
+from rest_framework import viewsets, status, generics
+from rest_framework.pagination import LimitOffsetPagination
+from rest_framework import filters
+from rest_framework_bulk import BulkModelViewSet
+from django.shortcuts import get_object_or_404
+
+from common.permissions import IsOrgAdminOrAppUser, NeedMFAVerify
+from common.utils import get_object_or_none, get_logger
+from common.mixins import IDInCacheFilterMixin
+from ..backends import AssetUserManager
+from ..models import Asset, Node, SystemUser, AdminUser
+from .. import serializers
+from ..tasks import test_asset_users_connectivity_manual
+
+
+__all__ = [
+    'AssetUserViewSet', 'AssetUserAuthInfoApi', 'AssetUserTestConnectiveApi',
+    'AssetUserExportViewSet',
+]
+
+
+logger = get_logger(__name__)
+
+
+class AssetUserFilterBackend(filters.BaseFilterBackend):
+    def filter_queryset(self, request, queryset, view):
+        kwargs = {}
+        for field in view.filter_fields:
+            value = request.GET.get(field)
+            if not value:
+                continue
+            if field in ("node_id", "system_user_id", "admin_user_id"):
+                continue
+            kwargs[field] = value
+        return queryset.filter(**kwargs)
+
+
+class AssetUserSearchBackend(filters.BaseFilterBackend):
+    def filter_queryset(self, request, queryset, view):
+        value = request.GET.get('search')
+        if not value:
+            return queryset
+        _queryset = AssetUserManager.none()
+        for field in view.search_fields:
+            if field in ("node_id", "system_user_id", "admin_user_id"):
+                continue
+            _queryset |= queryset.filter(**{field: value})
+        return _queryset
+
+
+class AssetUserViewSet(IDInCacheFilterMixin, BulkModelViewSet):
+    pagination_class = LimitOffsetPagination
+    serializer_class = serializers.AssetUserSerializer
+    permission_classes = [IsOrgAdminOrAppUser]
+    http_method_names = ['get', 'post']
+    filter_fields = [
+        "id", "ip", "hostname", "username", "asset_id", "node_id",
+        "system_user_id", "admin_user_id"
+    ]
+    search_fields = filter_fields
+    filter_backends = (
+        filters.OrderingFilter,
+        AssetUserFilterBackend, AssetUserSearchBackend,
+    )
+
+    def get_queryset(self):
+        # 尽可能先返回更少的数据
+        username = self.request.GET.get('username')
+        asset_id = self.request.GET.get('asset_id')
+        node_id = self.request.GET.get('node_id')
+        admin_user_id = self.request.GET.get("admin_user_id")
+        system_user_id = self.request.GET.get("system_user_id")
+
+        kwargs = {}
+        assets = None
+
+        manager = AssetUserManager()
+        if system_user_id:
+            system_user = get_object_or_404(SystemUser, id=system_user_id)
+            assets = system_user.assets.all()
+            username = system_user.username
+        elif admin_user_id:
+            admin_user = get_object_or_404(AdminUser, id=admin_user_id)
+            assets = admin_user.assets.all()
+            username = admin_user.username
+            manager.prefer('admin_user')
+
+        if asset_id:
+            asset = get_object_or_404(Asset, id=asset_id)
+            assets = [asset]
+        elif node_id:
+            node = get_object_or_404(Node, id=node_id)
+            assets = node.get_all_assets()
+
+        if username:
+            kwargs['username'] = username
+        if assets is not None:
+            kwargs['assets'] = assets
+
+        queryset = manager.filter(**kwargs)
+        return queryset
+
+
+class AssetUserExportViewSet(AssetUserViewSet):
+    serializer_class = serializers.AssetUserExportSerializer
+    http_method_names = ['get']
+    permission_classes = [IsOrgAdminOrAppUser, NeedMFAVerify]
+
+
+class AssetUserAuthInfoApi(generics.RetrieveAPIView):
+    serializer_class = serializers.AssetUserAuthInfoSerializer
+    permission_classes = [IsOrgAdminOrAppUser, NeedMFAVerify]
+
+    def retrieve(self, request, *args, **kwargs):
+        instance = self.get_object()
+        serializer = self.get_serializer(instance)
+        status_code = status.HTTP_200_OK
+        if not instance:
+            status_code = status.HTTP_400_BAD_REQUEST
+        return Response(serializer.data, status=status_code)
+
+    def get_object(self):
+        query_params = self.request.query_params
+        username = query_params.get('username')
+        asset_id = query_params.get('asset_id')
+        prefer = query_params.get("prefer")
+        asset = get_object_or_none(Asset, pk=asset_id)
+        try:
+            manger = AssetUserManager()
+            instance = manger.get(username, asset, prefer=prefer)
+        except Exception as e:
+            logger.error(e, exc_info=True)
+            return None
+        else:
+            return instance
+
+
+class AssetUserTestConnectiveApi(generics.RetrieveAPIView):
+    """
+    Test asset users connective
+    """
+    permission_classes = (IsOrgAdminOrAppUser,)
+    serializer_class = serializers.TaskIDSerializer
+
+    def get_asset_users(self):
+        username = self.request.GET.get('username')
+        asset_id = self.request.GET.get('asset_id')
+        prefer = self.request.GET.get("prefer")
+        asset = get_object_or_none(Asset, pk=asset_id)
+        manager = AssetUserManager()
+        asset_users = manager.filter(username=username, assets=[asset], prefer=prefer)
+        return asset_users
+
+    def retrieve(self, request, *args, **kwargs):
+        asset_users = self.get_asset_users()
+        prefer = self.request.GET.get("prefer")
+        kwargs = {}
+        if prefer == "admin_user":
+            kwargs["run_as_admin"] = True
+        task = test_asset_users_connectivity_manual.delay(asset_users, **kwargs)
+        return Response({"task": task.id})
+
+
+class AssetUserPushApi(generics.CreateAPIView):
+    """
+    Test asset users connective
+    """
+    serializer_class = serializers.AssetUserPushSerializer
+    permission_classes = (IsOrgAdminOrAppUser,)
+
+    def create(self, request, *args, **kwargs):
+        serializer = self.get_serializer(data=request.data)
+        serializer.is_valid(raise_exception=True)
+        asset = serializer.validated_data["asset"]
+        username = serializer.validated_data["username"]
+        pass

apps/assets/api/cmd_filter.py

@@ -0,0 +1,39 @@
+# -*- coding: utf-8 -*-
+#
+
+from rest_framework_bulk import BulkModelViewSet
+from rest_framework.pagination import LimitOffsetPagination
+from django.shortcuts import get_object_or_404
+
+from ..hands import IsOrgAdmin
+from ..models import CommandFilter, CommandFilterRule
+from .. import serializers
+
+
+__all__ = ['CommandFilterViewSet', 'CommandFilterRuleViewSet']
+
+
+class CommandFilterViewSet(BulkModelViewSet):
+    filter_fields = ("name",)
+    search_fields = filter_fields
+    permission_classes = (IsOrgAdmin,)
+    queryset = CommandFilter.objects.all()
+    serializer_class = serializers.CommandFilterSerializer
+    pagination_class = LimitOffsetPagination
+
+
+class CommandFilterRuleViewSet(BulkModelViewSet):
+    filter_fields = ("content",)
+    search_fields = filter_fields
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.CommandFilterRuleSerializer
+    pagination_class = LimitOffsetPagination
+
+    def get_queryset(self):
+        fpk = self.kwargs.get('filter_pk')
+        if not fpk:
+            return CommandFilterRule.objects.none()
+        cmd_filter = get_object_or_404(CommandFilter, pk=fpk)
+        return cmd_filter.rules.all()
+
+

apps/assets/api/domain.py

@@ -0,0 +1,61 @@
+# ~*~ coding: utf-8 ~*~
+
+from rest_framework_bulk import BulkModelViewSet
+from rest_framework.views import APIView, Response
+from rest_framework.pagination import LimitOffsetPagination
+
+from django.views.generic.detail import SingleObjectMixin
+
+from common.utils import get_logger
+from common.permissions import IsOrgAdmin, IsAppUser, IsOrgAdminOrAppUser
+from ..models import Domain, Gateway
+from .. import serializers
+
+
+logger = get_logger(__file__)
+__all__ = ['DomainViewSet', 'GatewayViewSet', "GatewayTestConnectionApi"]
+
+
+class DomainViewSet(BulkModelViewSet):
+    queryset = Domain.objects.all()
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.DomainSerializer
+    pagination_class = LimitOffsetPagination
+
+    def get_queryset(self):
+        queryset = super().get_queryset().all()
+        return queryset
+
+    def get_serializer_class(self):
+        if self.request.query_params.get('gateway'):
+            return serializers.DomainWithGatewaySerializer
+        return super().get_serializer_class()
+
+    def get_permissions(self):
+        if self.request.query_params.get('gateway'):
+            self.permission_classes = (IsOrgAdminOrAppUser,)
+        return super().get_permissions()
+
+
+class GatewayViewSet(BulkModelViewSet):
+    filter_fields = ("domain__name", "name", "username", "ip", "domain")
+    search_fields = filter_fields
+    queryset = Gateway.objects.all()
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.GatewaySerializer
+    pagination_class = LimitOffsetPagination
+
+
+class GatewayTestConnectionApi(SingleObjectMixin, APIView):
+    permission_classes = (IsOrgAdmin,)
+    model = Gateway
+    object = None
+
+    def post(self, request, *args, **kwargs):
+        self.object = self.get_object(Gateway.objects.all())
+        local_port = self.request.data.get('port') or self.object.port
+        ok, e = self.object.test_connective(local_port=local_port)
+        if ok:
+            return Response("ok")
+        else:
+            return Response({"failed": e}, status=404)

apps/assets/api/label.py

@@ -14,10 +14,11 @@
 # limitations under the License.
 
 from rest_framework_bulk import BulkModelViewSet
+from rest_framework.pagination import LimitOffsetPagination
 from django.db.models import Count
 
 from common.utils import get_logger
-from ..hands import IsSuperUser
+from ..hands import IsOrgAdmin
 from ..models import Label
 from .. import serializers
 
@@ -27,12 +28,18 @@ __all__ = ['LabelViewSet']
 
 
 class LabelViewSet(BulkModelViewSet):
-    queryset = Label.objects.annotate(asset_count=Count("assets"))
-    permission_classes = (IsSuperUser,)
+    filter_fields = ("name", "value")
+    search_fields = filter_fields
+    permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.LabelSerializer
+    pagination_class = LimitOffsetPagination
 
     def list(self, request, *args, **kwargs):
         if request.query_params.get("distinct"):
             self.serializer_class = serializers.LabelDistinctSerializer
             self.queryset = self.queryset.values("name").distinct()
         return super().list(request, *args, **kwargs)
+
+    def get_queryset(self):
+        self.queryset = Label.objects.annotate(asset_count=Count("assets"))
+        return self.queryset

apps/assets/api/node.py

@@ -13,32 +13,37 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from rest_framework import generics, mixins
+from rest_framework import generics, mixins, viewsets
+from rest_framework.serializers import ValidationError
 from rest_framework.views import APIView
 from rest_framework.response import Response
-from rest_framework_bulk import BulkModelViewSet
 from django.utils.translation import ugettext_lazy as _
 from django.shortcuts import get_object_or_404
 
 from common.utils import get_logger, get_object_or_none
-from ..hands import IsSuperUser
+from common.tree import TreeNodeSerializer
+from ..hands import IsOrgAdmin
 from ..models import Node
-from ..tasks import update_assets_hardware_info_util, test_asset_connectability_util
+from ..tasks import update_assets_hardware_info_util, test_asset_connectivity_util
 from .. import serializers
+from ..utils import NodeUtil
 
 
 logger = get_logger(__file__)
 __all__ = [
-    'NodeViewSet', 'NodeChildrenApi',
-    'NodeAddAssetsApi', 'NodeRemoveAssetsApi',
+    'NodeViewSet', 'NodeChildrenApi', 'NodeAssetsApi',
+    'NodeAddAssetsApi', 'NodeRemoveAssetsApi', 'NodeReplaceAssetsApi',
     'NodeAddChildrenApi', 'RefreshNodeHardwareInfoApi',
-    'TestNodeConnectiveApi'
+    'TestNodeConnectiveApi', 'NodeListAsTreeApi',
+    'NodeChildrenAsTreeApi', 'RefreshAssetsAmount',
 ]
 
 
-class NodeViewSet(BulkModelViewSet):
+class NodeViewSet(viewsets.ModelViewSet):
+    filter_fields = ('value', 'key', )
+    search_fields = filter_fields
     queryset = Node.objects.all()
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.NodeSerializer
 
     def perform_create(self, serializer):
@@ -46,42 +51,171 @@ class NodeViewSet(BulkModelViewSet):
         serializer.validated_data["key"] = child_key
         serializer.save()
 
+    def update(self, request, *args, **kwargs):
+        node = self.get_object()
+        if node.is_root():
+            node_value = node.value
+            post_value = request.data.get('value')
+            if node_value != post_value:
+                return Response(
+                    {"msg": _("You can't update the root node name")},
+                    status=400
+                )
+        return super().update(request, *args, **kwargs)
+
+
+class NodeListAsTreeApi(generics.ListAPIView):
+    """
+    获取节点列表树
+    [
+      {
+        "id": "",
+        "name": "",
+        "pId": "",
+        "meta": ""
+      }
+    ]
+    """
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = TreeNodeSerializer
+
+    def get_queryset(self):
+        queryset = Node.objects.all()
+        util = NodeUtil()
+        nodes = util.get_nodes_by_queryset(queryset)
+        queryset = [node.as_tree_node() for node in nodes]
+        return queryset
+
+    @staticmethod
+    def refresh_nodes(queryset):
+        Node.expire_nodes_assets_amount()
+        Node.expire_nodes_full_value()
+        return queryset
+
+
+class NodeChildrenAsTreeApi(generics.ListAPIView):
+    """
+    节点子节点作为树返回，
+    [
+      {
+        "id": "",
+        "name": "",
+        "pId": "",
+        "meta": ""
+      }
+    ]
+
+    """
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = TreeNodeSerializer
+    node = None
+    is_root = False
+
+    def get_queryset(self):
+        self.check_need_refresh_nodes()
+        node_key = self.request.query_params.get('key')
+        util = NodeUtil()
+        # 是否包含自己
+        with_self = False
+        if not node_key:
+            node_key = Node.root().key
+            with_self = True
+        self.node = util.get_node_by_key(node_key)
+        queryset = self.node.get_children(with_self=with_self)
+        queryset = [node.as_tree_node() for node in queryset]
+        queryset = sorted(queryset)
+        return queryset
+
+    def filter_assets(self, queryset):
+        include_assets = self.request.query_params.get('assets', '0') == '1'
+        if not include_assets:
+            return queryset
+        assets = self.node.get_assets().only(
+            "id", "hostname", "ip", 'platform', "os", "org_id",
+        )
+        for asset in assets:
+            queryset.append(asset.as_tree_node(self.node))
+        return queryset
+
+    def filter_queryset(self, queryset):
+        queryset = self.filter_assets(queryset)
+        return queryset
+
+    def check_need_refresh_nodes(self):
+        if self.request.query_params.get('refresh', '0') == '1':
+            Node.refresh_nodes()
+
 
 class NodeChildrenApi(mixins.ListModelMixin, generics.CreateAPIView):
     queryset = Node.objects.all()
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.NodeSerializer
     instance = None
 
+    def get(self, request, *args, **kwargs):
+        return self.list(request, *args, **kwargs)
+
     def post(self, request, *args, **kwargs):
+        instance = self.get_object()
         if not request.data.get("value"):
-            request.data["value"] = _("New node {}").format(
-                Node.root().get_next_child_key().split(":")[-1]
-            )
+            request.data["value"] = instance.get_next_child_preset_name()
         return super().post(request, *args, **kwargs)
 
     def create(self, request, *args, **kwargs):
         instance = self.get_object()
         value = request.data.get("value")
-        node = instance.create_child(value=value)
-        return Response(
-            {"id": node.id, "key": node.key, "value": node.value},
-            status=201,
-        )
+        _id = request.data.get('id') or None
+        values = [child.value for child in instance.get_children()]
+        if value in values:
+            raise ValidationError(
+                'The same level node name cannot be the same'
+            )
+        node = instance.create_child(value=value, _id=_id)
+        return Response(self.serializer_class(instance=node).data, status=201)
 
-    def get(self, request, *args, **kwargs):
-        instance = self.get_object()
-        if self.request.query_params.get("all"):
-            children = instance.get_all_children()
+    def get_object(self):
+        pk = self.kwargs.get('pk') or self.request.query_params.get('id')
+        if not pk:
+            node = Node.root()
         else:
-            children = instance.get_children()
-        response = [{"id": node.id, "key": node.key, "value": node.value} for node in children]
-        return Response(response, status=200)
+            node = get_object_or_404(Node, pk=pk)
+        return node
+
+    def get_queryset(self):
+        queryset = []
+        query_all = self.request.query_params.get("all")
+        node = self.get_object()
+
+        if node is None:
+            node = Node.root()
+            node.assets__count = node.get_all_assets().count()
+            queryset.append(node)
+
+        if query_all:
+            children = node.get_all_children()
+        else:
+            children = node.get_children()
+        queryset.extend(list(children))
+        return queryset
+
+
+class NodeAssetsApi(generics.ListAPIView):
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.AssetSerializer
+
+    def get_queryset(self):
+        node_id = self.kwargs.get('pk')
+        query_all = self.request.query_params.get('all')
+        instance = get_object_or_404(Node, pk=node_id)
+        if query_all:
+            return instance.get_all_assets()
+        else:
+            return instance.get_assets()
 
 
 class NodeAddChildrenApi(generics.UpdateAPIView):
     queryset = Node.objects.all()
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.NodeAddChildrenSerializer
     instance = None
 
@@ -93,14 +227,13 @@ class NodeAddChildrenApi(generics.UpdateAPIView):
             if not node:
                 continue
             node.parent = instance
-            node.save()
         return Response("OK")
 
 
 class NodeAddAssetsApi(generics.UpdateAPIView):
     serializer_class = serializers.NodeAssetsSerializer
     queryset = Node.objects.all()
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
     instance = None
 
     def perform_update(self, serializer):
@@ -112,7 +245,7 @@ class NodeAddAssetsApi(generics.UpdateAPIView):
 class NodeRemoveAssetsApi(generics.UpdateAPIView):
     serializer_class = serializers.NodeAssetsSerializer
     queryset = Node.objects.all()
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
     instance = None
 
     def perform_update(self, serializer):
@@ -120,31 +253,56 @@ class NodeRemoveAssetsApi(generics.UpdateAPIView):
         instance = self.get_object()
         if instance != Node.root():
             instance.assets.remove(*tuple(assets))
+        else:
+            assets = [asset for asset in assets if asset.nodes.count() > 1]
+            instance.assets.remove(*tuple(assets))
+
+
+class NodeReplaceAssetsApi(generics.UpdateAPIView):
+    serializer_class = serializers.NodeAssetsSerializer
+    queryset = Node.objects.all()
+    permission_classes = (IsOrgAdmin,)
+    instance = None
+
+    def perform_update(self, serializer):
+        assets = serializer.validated_data.get('assets')
+        instance = self.get_object()
+        for asset in assets:
+            asset.nodes.set([instance])
 
 
 class RefreshNodeHardwareInfoApi(APIView):
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
     model = Node
 
     def get(self, request, *args, **kwargs):
         node_id = kwargs.get('pk')
         node = get_object_or_404(self.model, id=node_id)
-        assets = node.assets.all()
-        # task_name = _("Refresh node assets hardware info: {}".format(node.name))
-        task_name = _("更新节点资产硬件信息: {}".format(node.name))
-        update_assets_hardware_info_util.delay(assets, task_name=task_name)
-        return Response({"msg": "Task created"})
+        assets = node.get_all_assets()
+        # task_name = _("更新节点资产硬件信息: {}".format(node.name))
+        task_name = _("Update node asset hardware information: {}").format(node.name)
+        task = update_assets_hardware_info_util.delay(assets, task_name=task_name)
+        return Response({"task": task.id})
 
 
 class TestNodeConnectiveApi(APIView):
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
     model = Node
 
     def get(self, request, *args, **kwargs):
         node_id = kwargs.get('pk')
         node = get_object_or_404(self.model, id=node_id)
-        assets = node.assets.all()
-        task_name = _("测试节点下资产是否可连接: {}".format(node.name))
-        test_asset_connectability_util.delay(assets, task_name=task_name)
-        return Response({"msg": "Task created"})
+        assets = node.get_all_assets()
+        # task_name = _("测试节点下资产是否可连接: {}".format(node.name))
+        task_name = _("Test if the assets under the node are connectable: {}".format(node.name))
+        task = test_asset_connectivity_util.delay(assets, task_name=task_name)
+        return Response({"task": task.id})
 
+
+class RefreshAssetsAmount(APIView):
+    permission_classes = (IsOrgAdmin,)
+    model = Node
+
+    def get(self, request, *args, **kwargs):
+        self.model.expire_nodes_assets_amount()
+        return Response("Ok")

apps/assets/api/system_user.py

@@ -13,49 +13,77 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from django.shortcuts import get_object_or_404
 from rest_framework import generics
 from rest_framework.response import Response
 from rest_framework_bulk import BulkModelViewSet
+from rest_framework.pagination import LimitOffsetPagination
+
 from common.utils import get_logger
-from ..hands import IsSuperUser, IsSuperUserOrAppUser
-from ..models import SystemUser
+from common.permissions import IsOrgAdmin, IsOrgAdminOrAppUser
+from common.mixins import IDInCacheFilterMixin
+from orgs.mixins import OrgBulkModelViewSet
+from ..models import SystemUser, Asset
 from .. import serializers
 from ..tasks import push_system_user_to_assets_manual, \
-    test_system_user_connectability_manual
+    test_system_user_connectivity_manual, push_system_user_a_asset_manual, \
+    test_system_user_connectivity_a_asset
 
 
 logger = get_logger(__file__)
 __all__ = [
-    'SystemUserViewSet', 'SystemUserAuthInfoApi',
-    'SystemUserPushApi', 'SystemUserTestConnectiveApi'
+    'SystemUserViewSet', 'SystemUserAuthInfoApi', 'SystemUserAssetAuthInfoApi',
+    'SystemUserPushApi', 'SystemUserTestConnectiveApi',
+    'SystemUserAssetsListView', 'SystemUserPushToAssetApi',
+    'SystemUserTestAssetConnectivityApi', 'SystemUserCommandFilterRuleListApi',
+
 ]
 
 
-class SystemUserViewSet(BulkModelViewSet):
+class SystemUserViewSet(OrgBulkModelViewSet):
     """
     System user api set, for add,delete,update,list,retrieve resource
     """
+    filter_fields = ("name", "username")
+    search_fields = filter_fields
     queryset = SystemUser.objects.all()
     serializer_class = serializers.SystemUserSerializer
-    permission_classes = (IsSuperUserOrAppUser,)
+    permission_classes = (IsOrgAdminOrAppUser,)
+    pagination_class = LimitOffsetPagination
+
+    def get_queryset(self):
+        queryset = super().get_queryset().all()
+        return queryset
 
 
-class SystemUserAuthInfoApi(generics.RetrieveUpdateAPIView):
+class SystemUserAuthInfoApi(generics.RetrieveUpdateDestroyAPIView):
     """
     Get system user auth info
     """
     queryset = SystemUser.objects.all()
-    permission_classes = (IsSuperUserOrAppUser,)
+    permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.SystemUserAuthSerializer
 
-    def update(self, request, *args, **kwargs):
-        password = request.data.pop("password", None)
-        private_key = request.data.pop("private_key", None)
+    def destroy(self, request, *args, **kwargs):
         instance = self.get_object()
+        instance.clear_auth()
+        return Response(status=204)
 
-        if password or private_key:
-            instance.set_auth(password=password, private_key=private_key)
-        return super().update(request, *args, **kwargs)
+
+class SystemUserAssetAuthInfoApi(generics.RetrieveAPIView):
+    """
+    Get system user with asset auth info
+    """
+    queryset = SystemUser.objects.all()
+    permission_classes = (IsOrgAdminOrAppUser,)
+    serializer_class = serializers.SystemUserAuthSerializer
+
+    def get_object(self):
+        instance = super().get_object()
+        aid = self.kwargs.get('aid')
+        asset = get_object_or_404(Asset, pk=aid)
+        instance.load_specific_asset_auth(asset)
+        return instance
 
 
 class SystemUserPushApi(generics.RetrieveAPIView):
@@ -63,12 +91,15 @@ class SystemUserPushApi(generics.RetrieveAPIView):
     Push system user to cluster assets api
     """
     queryset = SystemUser.objects.all()
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
 
     def retrieve(self, request, *args, **kwargs):
         system_user = self.get_object()
-        push_system_user_to_assets_manual.delay(system_user)
-        return Response({"msg": "Task created"})
+        nodes = system_user.nodes.all()
+        for node in nodes:
+            system_user.assets.add(*tuple(node.get_all_assets()))
+        task = push_system_user_to_assets_manual.delay(system_user)
+        return Response({"task": task.id})
 
 
 class SystemUserTestConnectiveApi(generics.RetrieveAPIView):
@@ -76,9 +107,65 @@ class SystemUserTestConnectiveApi(generics.RetrieveAPIView):
     Push system user to cluster assets api
     """
     queryset = SystemUser.objects.all()
-    permission_classes = (IsSuperUser,)
+    permission_classes = (IsOrgAdmin,)
 
     def retrieve(self, request, *args, **kwargs):
         system_user = self.get_object()
-        test_system_user_connectability_manual.delay(system_user)
-        return Response({"msg": "Task created"})
+        task = test_system_user_connectivity_manual.delay(system_user)
+        return Response({"task": task.id})
+
+
+class SystemUserAssetsListView(generics.ListAPIView):
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.AssetSimpleSerializer
+    pagination_class = LimitOffsetPagination
+    filter_fields = ("hostname", "ip")
+    http_method_names = ['get']
+    search_fields = filter_fields
+
+    def get_object(self):
+        pk = self.kwargs.get('pk')
+        return get_object_or_404(SystemUser, pk=pk)
+
+    def get_queryset(self):
+        system_user = self.get_object()
+        return system_user.assets.all()
+
+
+class SystemUserPushToAssetApi(generics.RetrieveAPIView):
+    queryset = SystemUser.objects.all()
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.TaskIDSerializer
+
+    def retrieve(self, request, *args, **kwargs):
+        system_user = self.get_object()
+        asset_id = self.kwargs.get('aid')
+        asset = get_object_or_404(Asset, id=asset_id)
+        task = push_system_user_a_asset_manual.delay(system_user, asset)
+        return Response({"task": task.id})
+
+
+class SystemUserTestAssetConnectivityApi(generics.RetrieveAPIView):
+    queryset = SystemUser.objects.all()
+    permission_classes = (IsOrgAdmin,)
+    serializer_class = serializers.TaskIDSerializer
+
+    def retrieve(self, request, *args, **kwargs):
+        system_user = self.get_object()
+        asset_id = self.kwargs.get('aid')
+        asset = get_object_or_404(Asset, id=asset_id)
+        task = test_system_user_connectivity_a_asset.delay(system_user, asset)
+        return Response({"task": task.id})
+
+
+class SystemUserCommandFilterRuleListApi(generics.ListAPIView):
+    permission_classes = (IsOrgAdminOrAppUser,)
+
+    def get_serializer_class(self):
+        from ..serializers import CommandFilterRuleSerializer
+        return CommandFilterRuleSerializer
+
+    def get_queryset(self):
+        pk = self.kwargs.get('pk', None)
+        system_user = get_object_or_404(SystemUser, pk=pk)
+        return system_user.cmd_filter_rules

apps/assets/backends/__init__.py

@@ -0,0 +1 @@
+from .manager import AssetUserManager

apps/assets/backends/admin_user.py

@@ -0,0 +1,10 @@
+# -*- coding: utf-8 -*-
+#
+
+from ..models import AdminUser
+from .asset_user import AssetUserBackend
+
+
+class AdminUserBackend(AssetUserBackend):
+    model = AdminUser
+    backend = 'AdminUser'

apps/assets/backends/asset_user.py

@@ -0,0 +1,40 @@
+# -*- coding: utf-8 -*-
+#
+from .base import BaseBackend
+
+
+class AssetUserBackend(BaseBackend):
+    model = None
+    backend = "AssetUser"
+
+    @classmethod
+    def filter_queryset_more(cls, queryset):
+        return queryset
+
+    @classmethod
+    def filter(cls, username=None, assets=None, **kwargs):
+        queryset = cls.model.objects.all()
+        prefer_id = kwargs.get('prefer_id')
+        if prefer_id:
+            queryset = queryset.filter(id=prefer_id)
+            instances = cls.construct_authbook_objects(queryset, assets)
+            return instances
+        if username:
+            queryset = queryset.filter(username=username)
+        if assets:
+            queryset = queryset.filter(assets__in=assets).distinct()
+        queryset = cls.filter_queryset_more(queryset)
+        instances = cls.construct_authbook_objects(queryset, assets)
+        return instances
+
+    @classmethod
+    def construct_authbook_objects(cls, asset_users, assets):
+        instances = []
+        for asset_user in asset_users:
+            if not assets:
+                assets = asset_user.assets.all()
+            for asset in assets:
+                instance = asset_user.construct_to_authbook(asset)
+                instance.backend = cls.backend
+                instances.append(instance)
+        return instances

apps/assets/backends/base.py

@@ -0,0 +1,86 @@
+# -*- coding: utf-8 -*-
+#
+import uuid
+from abc import abstractmethod
+
+
+class BaseBackend:
+    @classmethod
+    @abstractmethod
+    def filter(cls, username=None, assets=None, latest=True, prefer=None, prefer_id=None):
+        """
+        :param username: 用户名
+        :param assets: <Asset>对象
+        :param latest: 是否是最新记录
+        :param prefer: 优先使用
+        :param prefer_id: 使用id
+        :return: 元素为<AuthBook>的可迭代对象(<list> or <QuerySet>)
+        """
+        pass
+
+
+class AssetUserQuerySet(list):
+    def order_by(self, *ordering):
+        _ordering = []
+        reverse = False
+        for i in ordering:
+            if i[0] == '-':
+                reverse = True
+                i = i[1:]
+            _ordering.append(i)
+        self.sort(key=lambda obj: [getattr(obj, j) for j in _ordering], reverse=reverse)
+        return self
+
+    def filter_in(self, kwargs):
+        in_kwargs = {}
+        queryset = []
+        for k, v in kwargs.items():
+            if len(v) == 0:
+                return self
+            if k.find("__in") >= 0:
+                in_kwargs[k] = v
+        for k in in_kwargs:
+            kwargs.pop(k)
+
+        if len(in_kwargs) == 0:
+            return self
+        for i in self:
+            matched = True
+            for k, v in in_kwargs.items():
+                key = k.split('__')[0]
+                attr = getattr(i, key, None)
+                # 如果属性或者value中是uuid,则转换成string
+                if isinstance(v[0], uuid.UUID):
+                    v = [str(i) for i in v]
+                if isinstance(attr, uuid.UUID):
+                    attr = str(attr)
+                if attr not in v:
+                    matched = False
+            if matched:
+                queryset.append(i)
+        return AssetUserQuerySet(queryset)
+
+    def filter_equal(self, kwargs):
+        def filter_it(obj):
+            wanted = []
+            real = []
+            for k, v in kwargs.items():
+                wanted.append(v)
+                value = getattr(obj, k)
+                if isinstance(value, uuid.UUID):
+                    value = str(value)
+                real.append(value)
+            return wanted == real
+        if len(kwargs) > 0:
+            queryset = AssetUserQuerySet([i for i in self if filter_it(i)])
+        else:
+            queryset = self
+        return queryset
+
+    def filter(self, **kwargs):
+        queryset = self.filter_in(kwargs).filter_equal(kwargs)
+        return queryset
+
+    def __or__(self, other):
+        self.extend(other)
+        return self

apps/assets/backends/db.py

@@ -0,0 +1,29 @@
+# -*- coding: utf-8 -*-
+#
+
+from ..models import AuthBook
+from .base import BaseBackend
+
+
+class AuthBookBackend(BaseBackend):
+    @classmethod
+    def filter(cls, username=None, assets=None, latest=True, **kwargs):
+        queryset = AuthBook.objects.all()
+        if username is not None:
+            queryset = queryset.filter(username=username)
+        if assets:
+            queryset = queryset.filter(asset__in=assets)
+        if latest:
+            queryset = queryset.latest_version()
+        return queryset
+
+    @classmethod
+    def create(cls, **kwargs):
+        auth_info = {
+            'password': kwargs.pop('password', ''),
+            'public_key': kwargs.pop('public_key', ''),
+            'private_key': kwargs.pop('private_key', '')
+        }
+        obj = AuthBook.objects.create(**kwargs)
+        obj.set_auth(**auth_info)
+        return obj

apps/assets/backends/manager.py

@@ -0,0 +1,110 @@
+# -*- coding: utf-8 -*-
+#
+from django.core.exceptions import MultipleObjectsReturned, ObjectDoesNotExist
+
+from .base import AssetUserQuerySet
+from .db import AuthBookBackend
+from .system_user import SystemUserBackend
+from .admin_user import AdminUserBackend
+
+
+class NotSupportError(Exception):
+    pass
+
+
+class AssetUserManager:
+    """
+    资产用户管理器
+    """
+    ObjectDoesNotExist = ObjectDoesNotExist
+    MultipleObjectsReturned = MultipleObjectsReturned
+    NotSupportError = NotSupportError
+    MSG_NOT_EXIST = '{} Object matching query does not exist'
+    MSG_MULTIPLE = '{} get() returned more than one object ' \
+                   '-- it returned {}!'
+
+    backends = (
+        ('db', AuthBookBackend),
+        ('system_user', SystemUserBackend),
+        ('admin_user', AdminUserBackend),
+    )
+
+    _prefer = "system_user"
+
+    def filter(self, username=None, assets=None, latest=True, prefer=None, prefer_id=None):
+        if assets is not None and not assets:
+            return AssetUserQuerySet([])
+
+        if prefer:
+            self._prefer = prefer
+
+        instances_map = {}
+        instances = []
+        for name, backend in self.backends:
+            if name != "db" and self._prefer != name:
+                continue
+            _instances = backend.filter(
+                username=username, assets=assets, latest=latest,
+                prefer=self._prefer, prefer_id=prefer_id,
+            )
+            instances_map[name] = _instances
+
+        # 如果不是获取最新版本，就不再merge
+        if not latest:
+            for _instances in instances_map.values():
+                instances.extend(_instances)
+            return AssetUserQuerySet(instances)
+
+        # merge的顺序
+        ordering = ["db"]
+        if self._prefer == "system_user":
+            ordering.extend(["system_user", "admin_user"])
+        else:
+            ordering.extend(["admin_user", "system_user"])
+        # 根据prefer决定优先使用系统用户或管理用户谁的
+        ordering_instances = [instances_map.get(i, []) for i in ordering]
+        instances = self._merge_instances(*ordering_instances)
+        return AssetUserQuerySet(instances)
+
+    def get(self, username, asset, **kwargs):
+        instances = self.filter(username, assets=[asset], **kwargs)
+        if len(instances) == 1:
+            return instances[0]
+        elif len(instances) == 0:
+            self.raise_does_not_exist(self.__name__)
+        else:
+            self.raise_multiple_return(self.__name__, len(instances))
+
+    def raise_does_not_exist(self, name):
+        raise self.ObjectDoesNotExist(self.MSG_NOT_EXIST.format(name))
+
+    def raise_multiple_return(self, name, length):
+        raise self.MultipleObjectsReturned(self.MSG_MULTIPLE.format(name, length))
+
+    @staticmethod
+    def create(**kwargs):
+        instance = AuthBookBackend.create(**kwargs)
+        return instance
+
+    def all(self):
+        return self.filter()
+
+    def prefer(self, s):
+        self._prefer = s
+        return self
+
+    @staticmethod
+    def none():
+        return AssetUserQuerySet()
+
+    @staticmethod
+    def _merge_instances(*args):
+        instances = list(args[0])
+        keywords = [obj.keyword for obj in instances]
+
+        for _instances in args[1:]:
+            need_merge_instances = [obj for obj in _instances if obj.keyword not in keywords]
+            need_merge_keywords = [obj.keyword for obj in need_merge_instances]
+            instances.extend(need_merge_instances)
+            keywords.extend(need_merge_keywords)
+        return instances

apps/assets/backends/system_user.py

@@ -0,0 +1,30 @@
+# -*- coding: utf-8 -*-
+#
+
+import itertools
+
+from assets.models import SystemUser
+from .asset_user import AssetUserBackend
+
+
+class SystemUserBackend(AssetUserBackend):
+    model = SystemUser
+    backend = 'SystemUser'
+
+    @classmethod
+    def filter_queryset_more(cls, queryset):
+        queryset = cls._distinct_system_users_by_username(queryset)
+        return queryset
+
+    @classmethod
+    def _distinct_system_users_by_username(cls, system_users):
+        system_users = sorted(
+            system_users,
+            key=lambda su: (su.username, su.priority, su.date_updated),
+            reverse=True,
+        )
+        results = itertools.groupby(system_users, key=lambda su: su.username)
+        system_users = [next(result[1]) for result in results]
+        return system_users
+
+

apps/assets/backends/utils.py

@@ -0,0 +1,16 @@
+# -*- coding: utf-8 -*-
+#
+
+# from django.conf import settings
+
+from .db import AuthBookBackend
+# from .vault import VaultBackend
+
+
+def get_backend():
+    default_backend = AuthBookBackend
+
+    # if settings.BACKEND_ASSET_USER_AUTH_VAULT:
+    #     return VaultBackend
+
+    return default_backend

apps/assets/backends/vault.py

@@ -0,0 +1,11 @@
+# -*- coding: utf-8 -*-
+#
+
+from .base import BaseBackend
+
+
+class VaultBackend(BaseBackend):
+
+    @classmethod
+    def filter(cls, username=None, asset=None, latest=True):
+        pass

apps/assets/const.py

@@ -1,5 +1,6 @@
 # -*- coding: utf-8 -*-
 #
+from django.utils.translation import ugettext_lazy as _
 
 UPDATE_ASSETS_HARDWARE_TASKS = [
    {
@@ -10,7 +11,6 @@ UPDATE_ASSETS_HARDWARE_TASKS = [
    }
 ]
 
-ADMIN_USER_CONN_CACHE_KEY = "ADMIN_USER_CONN_{}"
 TEST_ADMIN_USER_CONN_TASKS = [
     {
         "name": "ping",
@@ -19,6 +19,14 @@ TEST_ADMIN_USER_CONN_TASKS = [
         }
     }
 ]
+TEST_WINDOWS_ADMIN_USER_CONN_TASKS = [
+    {
+        "name": "ping",
+        "action": {
+            "module": "win_ping",
+        }
+    }
+]
 
 ASSET_ADMIN_CONN_CACHE_KEY = "ASSET_ADMIN_USER_CONN_{}"
 
@@ -31,8 +39,43 @@ TEST_SYSTEM_USER_CONN_TASKS = [
        }
    }
 ]
+TEST_WINDOWS_SYSTEM_USER_CONN_TASKS = [
+    {
+        "name": "ping",
+        "action": {
+            "module": "win_ping",
+        }
+    }
+]
+
+TEST_ASSET_USER_CONN_TASKS = [
+    {
+        "name": "ping",
+        "action": {
+            "module": "ping",
+        }
+    }
+]
+TEST_WINDOWS_ASSET_USER_CONN_TASKS = [
+    {
+        "name": "ping",
+        "action": {
+            "module": "win_ping",
+        }
+    }
+]
+
 
 TASK_OPTIONS = {
     'timeout': 10,
     'forks': 10,
 }
+
+CACHE_KEY_ASSET_BULK_UPDATE_ID_PREFIX = '_KEY_ASSET_BULK_UPDATE_ID_{}'
+CONN_UNREACHABLE, CONN_REACHABLE, CONN_UNKNOWN = range(0, 3)
+CONNECTIVITY_CHOICES = (
+    (CONN_UNREACHABLE, _("Unreachable")),
+    (CONN_REACHABLE, _('Reachable')),
+    (CONN_UNKNOWN, _("Unknown")),
+)
+

apps/assets/forms/__init__.py

@@ -3,3 +3,5 @@
 from .asset import *
 from .label import *
 from .user import *
+from .domain import *
+from .cmd_filter import *

apps/assets/forms/asset.py

@@ -3,20 +3,42 @@
 from django import forms
 from django.utils.translation import gettext_lazy as _
 
-from ..models import Asset, AdminUser
 from common.utils import get_logger
+from orgs.mixins import OrgModelForm
+
+from ..models import Asset, Node
+
 
 logger = get_logger(__file__)
-__all__ = ['AssetCreateForm', 'AssetUpdateForm', 'AssetBulkUpdateForm']
+__all__ = [
+    'AssetCreateForm', 'AssetUpdateForm', 'AssetBulkUpdateForm', 'ProtocolForm',
+]
 
 
-class AssetCreateForm(forms.ModelForm):
+class ProtocolForm(forms.Form):
+    name = forms.ChoiceField(
+        choices=Asset.PROTOCOL_CHOICES, label=_("Name"), initial='ssh',
+        widget=forms.Select(attrs={'class': 'form-control protocol-name'})
+    )
+    port = forms.IntegerField(
+        max_value=65534, min_value=1, label=_("Port"), initial=22,
+        widget=forms.TextInput(attrs={'class': 'form-control protocol-port'})
+    )
+
+
+class AssetCreateForm(OrgModelForm):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        if not self.data:
+            nodes_field = self.fields['nodes']
+            nodes_field._queryset = Node.get_queryset()
+
     class Meta:
         model = Asset
         fields = [
-            'hostname', 'ip', 'public_ip', 'port',  'comment',
+            'hostname', 'ip', 'public_ip', 'protocols', 'comment',
             'nodes', 'is_active', 'admin_user', 'labels', 'platform',
-
+            'domain',
         ]
         widgets = {
             'nodes': forms.SelectMultiple(attrs={
@@ -26,57 +48,63 @@ class AssetCreateForm(forms.ModelForm):
                 'class': 'select2', 'data-placeholder': _('Admin user')
             }),
             'labels': forms.SelectMultiple(attrs={
-                'class': 'select2', 'data-placeholder': _('Labels')
+                'class': 'select2', 'data-placeholder': _('Label')
             }),
-            'port': forms.TextInput(),
+            'domain': forms.Select(attrs={
+                'class': 'select2', 'data-placeholder': _('Domain')
+            }),
+        }
+        labels = {
+            'nodes': _("Node"),
         }
         help_texts = {
-            'hostname': '* required',
-            'ip': '* required',
-            'port': '* required',
             'admin_user': _(
                 'root or other NOPASSWD sudo privilege user existed in asset,'
                 'If asset is windows or other set any one, more see admin user left menu'
             ),
-            'platform': _("* required Must set exact system platform, Windows, Linux ...")
+            'platform': _("Windows 2016 RDP protocol is different, If is window 2016, set it"),
+            'domain': _("If your have some network not connect with each other, you can set domain")
         }
 
 
-class AssetUpdateForm(forms.ModelForm):
+class AssetUpdateForm(OrgModelForm):
     class Meta:
         model = Asset
         fields = [
-            'hostname', 'ip', 'port', 'nodes',  'is_active', 'platform',
+            'hostname', 'ip', 'protocols', 'nodes',  'is_active', 'platform',
             'public_ip', 'number', 'comment', 'admin_user', 'labels',
+            'domain',
         ]
         widgets = {
             'nodes': forms.SelectMultiple(attrs={
-                'class': 'select2', 'data-placeholder': _('Nodes')
+                'class': 'select2', 'data-placeholder': _('Node')
             }),
             'admin_user': forms.Select(attrs={
                 'class': 'select2', 'data-placeholder': _('Admin user')
             }),
             'labels': forms.SelectMultiple(attrs={
-                'class': 'select2', 'data-placeholder': _('Labels')
+                'class': 'select2', 'data-placeholder': _('Label')
             }),
-            'port': forms.TextInput(),
+            'domain': forms.Select(attrs={
+                'class': 'select2', 'data-placeholder': _('Domain')
+            }),
+        }
+        labels = {
+            'nodes': _("Node"),
         }
         help_texts = {
-            'hostname': '* required',
-            'ip': '* required',
-            'port': '* required',
-            'cluster': '* required',
             'admin_user': _(
                 'root or other NOPASSWD sudo privilege user existed in asset,'
                 'If asset is windows or other set any one, more see admin user left menu'
             ),
-            'platform': _("* required Must set exact system platform, Windows, Linux ...")
+            'platform': _("Windows 2016 RDP protocol is different, If is window 2016, set it"),
+            'domain': _("If your have some network not connect with each other, you can set domain")
         }
 
 
-class AssetBulkUpdateForm(forms.ModelForm):
+class AssetBulkUpdateForm(OrgModelForm):
     assets = forms.ModelMultipleChoiceField(
-        required=True, help_text='* required',
+        required=True,
         label=_('Select assets'), queryset=Asset.objects.all(),
         widget=forms.SelectMultiple(
             attrs={
@@ -85,34 +113,29 @@ class AssetBulkUpdateForm(forms.ModelForm):
             }
         )
     )
-    port = forms.IntegerField(
-        label=_('Port'), required=False, min_value=1, max_value=65535,
-    )
-    admin_user = forms.ModelChoiceField(
-        required=False, queryset=AdminUser.objects.all(),
-        label=_("Admin user"),
-        widget=forms.Select(
-            attrs={
-                'class': 'select2',
-                'data-placeholder': _('Admin user')
-            }
-        )
-    )
 
     class Meta:
         model = Asset
         fields = [
-            'assets', 'port',  'admin_user', 'labels', 'nodes', 'platform'
+            'assets', 'admin_user', 'labels', 'platform',
+            'domain',
         ]
         widgets = {
             'labels': forms.SelectMultiple(
-                attrs={'class': 'select2', 'data-placeholder': _('Select labels')}
+                attrs={'class': 'select2', 'data-placeholder': _('Label')}
             ),
             'nodes': forms.SelectMultiple(
-                attrs={'class': 'select2', 'data-placeholder': _('Select nodes')}
+                attrs={'class': 'select2', 'data-placeholder': _('Node')}
             ),
         }
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        # 重写其他字段为不再required
+        for name, field in self.fields.items():
+            if name != 'assets':
+                field.required = False
+
     def save(self, commit=True):
         changed_fields = []
         for field in self._meta.fields:
@@ -123,14 +146,14 @@ class AssetBulkUpdateForm(forms.ModelForm):
                         if k in changed_fields}
         assets = cleaned_data.pop('assets')
         labels = cleaned_data.pop('labels', [])
-        nodes = cleaned_data.pop('nodes')
+        nodes = cleaned_data.pop('nodes', None)
         assets = Asset.objects.filter(id__in=[asset.id for asset in assets])
         assets.update(**cleaned_data)
 
         if labels:
-            for label in labels:
-                label.assets.add(*tuple(assets))
+            for asset in assets:
+                asset.labels.set(labels)
         if nodes:
-            for node in nodes:
-                node.assets.add(*tuple(assets))
+            for asset in assets:
+                asset.nodes.set(nodes)
         return assets

apps/assets/forms/cmd_filter.py

@@ -0,0 +1,39 @@
+# -*- coding: utf-8 -*-
+#
+from django import forms
+from django.core.exceptions import ValidationError
+import re
+
+from orgs.mixins import OrgModelForm
+from ..models import CommandFilter, CommandFilterRule
+
+__all__ = ['CommandFilterForm', 'CommandFilterRuleForm']
+
+
+class CommandFilterForm(OrgModelForm):
+    class Meta:
+        model = CommandFilter
+        fields = ['name', 'comment']
+
+
+class CommandFilterRuleForm(OrgModelForm):
+    invalid_pattern = re.compile(r'[\.\*\+\[\\\?\{\}\^\$\|\(\)\#\<\>]')
+
+    class Meta:
+        model = CommandFilterRule
+        fields = [
+            'filter', 'type', 'content', 'priority', 'action', 'comment'
+        ]
+        widgets = {
+            'content':  forms.Textarea(attrs={
+                'placeholder': 'eg:\r\nreboot\r\nrm -rf'
+            }),
+        }
+
+    def clean_content(self):
+        content = self.cleaned_data.get("content")
+        if self.invalid_pattern.search(content):
+            invalid_char = self.invalid_pattern.pattern.replace('\\', '')
+            msg = _("Content should not be contain: {}").format(invalid_char)
+            raise ValidationError(msg)
+        return content

apps/assets/forms/domain.py

@@ -0,0 +1,75 @@
+# -*- coding: utf-8 -*-
+#
+from django import forms
+from django.utils.translation import gettext_lazy as _
+
+from orgs.mixins import OrgModelForm
+from ..models import Domain, Asset, Gateway
+from .user import PasswordAndKeyAuthForm
+
+__all__ = ['DomainForm', 'GatewayForm']
+
+
+class DomainForm(forms.ModelForm):
+    assets = forms.ModelMultipleChoiceField(
+        queryset=Asset.objects.all(), label=_('Asset'), required=False,
+        widget=forms.SelectMultiple(
+            attrs={'class': 'select2', 'data-placeholder': _('Select assets')}
+        )
+    )
+
+    class Meta:
+        model = Domain
+        fields = ['name', 'comment', 'assets']
+
+    def __init__(self, *args, **kwargs):
+        if kwargs.get('instance', None):
+            initial = kwargs.get('initial', {})
+            initial['assets'] = kwargs['instance'].assets.all()
+        super().__init__(*args, **kwargs)
+
+        # 前端渲染优化, 防止过多资产
+        assets_field = self.fields.get('assets')
+        if not self.data:
+            instance = kwargs.get('instance')
+            if instance:
+                assets_field.queryset = instance.assets.all()
+            else:
+                assets_field.queryset = Asset.objects.none()
+
+    def save(self, commit=True):
+        instance = super().save(commit=commit)
+        assets = self.cleaned_data['assets']
+        instance.assets.set(assets)
+        return instance
+
+
+class GatewayForm(PasswordAndKeyAuthForm, OrgModelForm):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        password_field = self.fields.get('password')
+        password_field.help_text = _('Password should not contain special characters')
+        protocol_field = self.fields.get('protocol')
+        protocol_field.choices = [Gateway.PROTOCOL_CHOICES[0]]
+
+    def save(self, commit=True):
+        # Because we define custom field, so we need rewrite :method: `save`
+        instance = super().save()
+        password = self.cleaned_data.get('password')
+        private_key, public_key = super().gen_keys()
+        instance.set_auth(password=password, private_key=private_key)
+        return instance
+
+    class Meta:
+        model = Gateway
+        fields = [
+            'name', 'ip', 'port', 'username', 'protocol', 'domain', 'password',
+            'private_key_file',  'is_active', 'comment',
+        ]
+        help_texts = {
+            'protocol': _("SSH gateway support proxy SSH,RDP,VNC")
+        }
+        widgets = {
+            'name': forms.TextInput(attrs={'placeholder': _('Name')}),
+            'username': forms.TextInput(attrs={'placeholder': _('Username')}),
+        }

apps/assets/forms/label.py

@@ -26,6 +26,15 @@ class LabelForm(forms.ModelForm):
             initial['assets'] = kwargs['instance'].assets.all()
         super().__init__(*args, **kwargs)
 
+        # 前端渲染优化, 防止过多资产
+        assets_field = self.fields.get('assets')
+        if not self.data:
+            instance = kwargs.get('instance')
+            if instance:
+                assets_field.queryset = instance.assets.all()
+            else:
+                assets_field.queryset = Asset.objects.none()
+
     def save(self, commit=True):
         label = super().save(commit=commit)
         assets = self.cleaned_data['assets']

apps/assets/forms/user.py

@@ -3,12 +3,13 @@
 from django import forms
 from django.utils.translation import gettext_lazy as _
 
-from ..models import AdminUser, SystemUser
 from common.utils import validate_ssh_private_key, ssh_pubkey_gen, get_logger
+from orgs.mixins import OrgModelForm
+from ..models import AdminUser, SystemUser
 
 logger = get_logger(__file__)
 __all__ = [
-    'FileForm', 'SystemUserForm', 'AdminUserForm',
+    'FileForm', 'SystemUserForm', 'AdminUserForm', 'PasswordAndKeyAuthForm',
 ]
 
 
@@ -34,8 +35,12 @@ class PasswordAndKeyAuthForm(forms.ModelForm):
         if private_key_file:
             key_string = private_key_file.read()
             private_key_file.seek(0)
+            key_string = key_string.decode()
+
             if not validate_ssh_private_key(key_string, password):
-                raise forms.ValidationError(_('Invalid private key'))
+                msg = _('Invalid private key, Only support '
+                        'RSA/DSA format key')
+                raise forms.ValidationError(msg)
         return private_key_file
 
     def validate_password_key(self):
@@ -79,13 +84,9 @@ class AdminUserForm(PasswordAndKeyAuthForm):
             'name': forms.TextInput(attrs={'placeholder': _('Name')}),
             'username': forms.TextInput(attrs={'placeholder': _('Username')}),
         }
-        help_texts = {
-            'name': '* required',
-            'username': '* required',
-        }
 
 
-class SystemUserForm(PasswordAndKeyAuthForm):
+class SystemUserForm(OrgModelForm, PasswordAndKeyAuthForm):
     # Admin user assets define, let user select, save it in form not in view
     auto_generate_key = forms.BooleanField(initial=True, required=False)
 
@@ -93,14 +94,28 @@ class SystemUserForm(PasswordAndKeyAuthForm):
         # Because we define custom field, so we need rewrite :method: `save`
         system_user = super().save()
         password = self.cleaned_data.get('password', '') or None
+        login_mode = self.cleaned_data.get('login_mode', '') or None
+        protocol = self.cleaned_data.get('protocol') or None
         auto_generate_key = self.cleaned_data.get('auto_generate_key', False)
         private_key, public_key = super().gen_keys()
 
+        if login_mode == SystemUser.LOGIN_MANUAL or \
+                protocol in [SystemUser.PROTOCOL_TELNET,
+                             SystemUser.PROTOCOL_VNC]:
+            system_user.auto_push = 0
+            system_user.save()
+            auto_generate_key = False
+
         if auto_generate_key:
             logger.info('Auto generate key and set system user auth')
-            system_user.auto_gen_auth()
+            if protocol == SystemUser.PROTOCOL_SSH:
+                system_user.auto_gen_auth()
+            elif protocol == SystemUser.PROTOCOL_RDP:
+                system_user.auto_gen_auth_password()
         else:
-            system_user.set_auth(password=password, private_key=private_key, public_key=public_key)
+            system_user.set_auth(password=password, private_key=private_key,
+                                 public_key=public_key)
+
         return system_user
 
     def clean(self):
@@ -109,27 +124,38 @@ class SystemUserForm(PasswordAndKeyAuthForm):
         if not self.instance and not auto_generate:
             super().validate_password_key()
 
+    def clean_username(self):
+        username = self.data.get('username')
+        login_mode = self.data.get('login_mode')
+        protocol = self.data.get('protocol')
+
+        if username:
+            return username
+        if login_mode == SystemUser.LOGIN_AUTO and \
+                protocol != SystemUser.PROTOCOL_VNC:
+            msg = _('* Automatic login mode must fill in the username.')
+            raise forms.ValidationError(msg)
+        return username
+
     class Meta:
         model = SystemUser
         fields = [
             'name', 'username', 'protocol', 'auto_generate_key',
             'password', 'private_key_file', 'auto_push', 'sudo',
-            'comment', 'shell', 'nodes', 'priority',
+            'comment', 'shell', 'priority', 'login_mode', 'cmd_filters',
         ]
         widgets = {
             'name': forms.TextInput(attrs={'placeholder': _('Name')}),
             'username': forms.TextInput(attrs={'placeholder': _('Username')}),
-            'nodes': forms.SelectMultiple(
-                attrs={
-                    'class': 'select2',
-                    'data-placeholder': _('Nodes')
-                }
-            ),
+            'cmd_filters': forms.SelectMultiple(attrs={
+                'class': 'select2', 'data-placeholder': _('Command filter')
+            }),
         }
         help_texts = {
-            'name': '* required',
-            'username': '* required',
-            'nodes': _('If auto push checked, system user will be create at node assets'),
             'auto_push': _('Auto push system user to asset'),
-            'priority': _('High level will be using login asset as default, if user was granted more than 2 system user'),
-        }
+            'priority': _('1-100, High level will be using login asset as default, '
+                          'if user was granted more than 2 system user'),
+            'login_mode': _('If you choose manual login mode, you do not '
+                            'need to fill in the username and password.'),
+            'sudo': _("Use comma split multi command, ex: /bin/whoami,/bin/ifconfig")
+        }

apps/assets/hands.py

@@ -11,7 +11,5 @@
 """
 
 
-from common.mixins import AdminUserRequiredMixin
-from common.permissions import IsAppUser, IsSuperUser, IsValidUser, IsSuperUserOrAppUser
+from common.permissions import IsAppUser, IsOrgAdmin, IsValidUser, IsOrgAdminOrAppUser
 from users.models import User, UserGroup
-from perms.utils import NodePermissionUtil

apps/assets/migrations/0002_auto_20180105_1807_squashed_0009_auto_20180307_1212.py

@@ -0,0 +1,158 @@
+# Generated by Django 2.1.7 on 2019-02-28 10:16
+
+import assets.models.asset
+from django.db import migrations, models
+import django.db.models.deletion
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    replaces = [('assets', '0002_auto_20180105_1807'), ('assets', '0003_auto_20180109_2331'), ('assets', '0004_auto_20180125_1218'), ('assets', '0005_auto_20180126_1637'), ('assets', '0006_auto_20180130_1502'), ('assets', '0007_auto_20180225_1815'), ('assets', '0008_auto_20180306_1804'), ('assets', '0009_auto_20180307_1212')]
+
+    dependencies = [
+        ('assets', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='adminuser',
+            options={'ordering': ['name'], 'verbose_name': 'Admin user'},
+        ),
+        migrations.AlterModelOptions(
+            name='asset',
+            options={'verbose_name': 'Asset'},
+        ),
+        migrations.AlterModelOptions(
+            name='assetgroup',
+            options={'ordering': ['name'], 'verbose_name': 'Asset group'},
+        ),
+        migrations.AlterModelOptions(
+            name='cluster',
+            options={'ordering': ['name'], 'verbose_name': 'Cluster'},
+        ),
+        migrations.AlterModelOptions(
+            name='systemuser',
+            options={'ordering': ['name'], 'verbose_name': 'System user'},
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='cluster',
+        ),
+        migrations.AlterField(
+            model_name='assetgroup',
+            name='created_by',
+            field=models.CharField(blank=True, max_length=32, null=True, verbose_name='Created by'),
+        ),
+        migrations.CreateModel(
+            name='Label',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(max_length=128, verbose_name='Name')),
+                ('value', models.CharField(max_length=128, verbose_name='Value')),
+                ('category', models.CharField(choices=[('S', 'System'), ('U', 'User')], default='U', max_length=128, verbose_name='Category')),
+                ('is_active', models.BooleanField(default=True, verbose_name='Is active')),
+                ('comment', models.TextField(blank=True, null=True, verbose_name='Comment')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+            ],
+            options={
+                'db_table': 'assets_label',
+            },
+        ),
+        migrations.AlterUniqueTogether(
+            name='label',
+            unique_together={('name', 'value')},
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='labels',
+            field=models.ManyToManyField(blank=True, related_name='assets', to='assets.Label', verbose_name='Labels'),
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='cabinet_no',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='cabinet_pos',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='env',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='remote_card_ip',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='status',
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='type',
+        ),
+        migrations.CreateModel(
+            name='Node',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('key', models.CharField(max_length=64, unique=True, verbose_name='Key')),
+                ('value', models.CharField(max_length=128, verbose_name='Value')),
+                ('child_mark', models.IntegerField(default=0)),
+                ('date_create', models.DateTimeField(auto_now_add=True)),
+            ],
+        ),
+        migrations.RemoveField(
+            model_name='asset',
+            name='groups',
+        ),
+        migrations.RemoveField(
+            model_name='systemuser',
+            name='cluster',
+        ),
+        migrations.AlterField(
+            model_name='asset',
+            name='admin_user',
+            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.PROTECT, to='assets.AdminUser', verbose_name='Admin user'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='protocol',
+            field=models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp')], default='ssh', max_length=16, verbose_name='Protocol'),
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='nodes',
+            field=models.ManyToManyField(default=assets.models.asset.default_node, related_name='assets', to='assets.Node', verbose_name='Nodes'),
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='nodes',
+            field=models.ManyToManyField(blank=True, to='assets.Node', verbose_name='Nodes'),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='created_by',
+            field=models.CharField(max_length=128, null=True, verbose_name='Created by'),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='username',
+            field=models.CharField(max_length=128, verbose_name='Username'),
+        ),
+        migrations.AlterField(
+            model_name='asset',
+            name='platform',
+            field=models.CharField(choices=[('Linux', 'Linux'), ('Unix', 'Unix'), ('MacOS', 'MacOS'), ('BSD', 'BSD'), ('Windows', 'Windows'), ('Other', 'Other')], default='Linux', max_length=128, verbose_name='Platform'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='created_by',
+            field=models.CharField(max_length=128, null=True, verbose_name='Created by'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='username',
+            field=models.CharField(max_length=128, verbose_name='Username'),
+        ),
+    ]

apps/assets/migrations/0010_auto_20180307_1749_squashed_0019_auto_20180816_1320.py

@@ -0,0 +1,220 @@
+# Generated by Django 2.1.7 on 2019-02-28 10:16
+
+import assets.models.utils
+import django.core.validators
+from django.db import migrations, models
+import django.db.models.deletion
+import uuid
+
+
+# Functions from the following migrations need manual copying.
+# Move them and any dependencies into this file, then update the
+# RunPython operations to refer to the local versions:
+# assets.migrations.0017_auto_20180702_1415
+
+def migrate_win_to_ssh_protocol(apps, schema_editor):
+    asset_model = apps.get_model("assets", "Asset")
+    db_alias = schema_editor.connection.alias
+    asset_model.objects.using(db_alias).filter(platform__startswith='Win').update(protocol='rdp')
+
+
+class Migration(migrations.Migration):
+
+    replaces = [('assets', '0010_auto_20180307_1749'), ('assets', '0011_auto_20180326_0957'), ('assets', '0012_auto_20180404_1302'), ('assets', '0013_auto_20180411_1135'), ('assets', '0014_auto_20180427_1245'), ('assets', '0015_auto_20180510_1235'), ('assets', '0016_auto_20180511_1203'), ('assets', '0017_auto_20180702_1415'), ('assets', '0018_auto_20180807_1116'), ('assets', '0019_auto_20180816_1320')]
+
+    dependencies = [
+        ('assets', '0009_auto_20180307_1212'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='node',
+            name='value',
+            field=models.CharField(max_length=128, unique=True, verbose_name='Value'),
+        ),
+        migrations.CreateModel(
+            name='Domain',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(max_length=128, unique=True, verbose_name='Name')),
+                ('comment', models.TextField(blank=True, verbose_name='Comment')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+            ],
+        ),
+        migrations.CreateModel(
+            name='Gateway',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(max_length=128, unique=True, verbose_name='Name')),
+                ('username', models.CharField(blank=True, max_length=32, validators=[django.core.validators.RegexValidator('^[0-9a-zA-Z_@\\-\\.]*$', 'Special char not allowed')], verbose_name='Username')),
+                ('_password', models.CharField(blank=True, max_length=256, null=True, verbose_name='Password')),
+                ('_private_key', models.TextField(blank=True, max_length=4096, null=True, validators=[assets.models.utils.private_key_validator], verbose_name='SSH private key')),
+                ('_public_key', models.TextField(blank=True, max_length=4096, verbose_name='SSH public key')),
+                ('date_created', models.DateTimeField(auto_now_add=True)),
+                ('date_updated', models.DateTimeField(auto_now=True)),
+                ('created_by', models.CharField(max_length=128, null=True, verbose_name='Created by')),
+                ('ip', models.GenericIPAddressField(db_index=True, verbose_name='IP')),
+                ('port', models.IntegerField(default=22, verbose_name='Port')),
+                ('protocol', models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp')], default='ssh', max_length=16, verbose_name='Protocol')),
+                ('comment', models.CharField(blank=True, max_length=128, null=True, verbose_name='Comment')),
+                ('is_active', models.BooleanField(default=True, verbose_name='Is active')),
+                ('domain', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='assets.Domain', verbose_name='Domain')),
+            ],
+            options={
+                'abstract': False,
+            },
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='domain',
+            field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='assets', to='assets.Domain', verbose_name='Domain'),
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='assets',
+            field=models.ManyToManyField(blank=True, to='assets.Asset', verbose_name='Assets'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='sudo',
+            field=models.TextField(default='/bin/whoami', verbose_name='Sudo'),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='username',
+            field=models.CharField(max_length=32, validators=[django.core.validators.RegexValidator('^[0-9a-zA-Z_-]*$', 'Special char not allowed')], verbose_name='Username'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='username',
+            field=models.CharField(max_length=32, validators=[django.core.validators.RegexValidator('^[0-9a-zA-Z_-]*$', 'Special char not allowed')], verbose_name='Username'),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='username',
+            field=models.CharField(max_length=32, validators=[django.core.validators.RegexValidator('^[0-9a-zA-Z_@\\-\\.]*$', 'Special char not allowed')], verbose_name='Username'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='username',
+            field=models.CharField(max_length=32, validators=[django.core.validators.RegexValidator('^[0-9a-zA-Z_@\\-\\.]*$', 'Special char not allowed')], verbose_name='Username'),
+        ),
+        migrations.AlterField(
+            model_name='node',
+            name='value',
+            field=models.CharField(max_length=128, verbose_name='Value'),
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='protocol',
+            field=models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp'), ('telnet', 'telnet (beta)')], default='ssh', max_length=128, verbose_name='Protocol'),
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='login_mode',
+            field=models.CharField(choices=[('auto', 'Automatic login'), ('manual', 'Manually login')], default='auto', max_length=10, verbose_name='Login mode'),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='username',
+            field=models.CharField(blank=True, max_length=32, validators=[django.core.validators.RegexValidator('^[0-9a-zA-Z_@\\-\\.]*$', 'Special char not allowed')], verbose_name='Username'),
+        ),
+        migrations.AlterField(
+            model_name='asset',
+            name='platform',
+            field=models.CharField(choices=[('Linux', 'Linux'), ('Unix', 'Unix'), ('MacOS', 'MacOS'), ('BSD', 'BSD'), ('Windows', 'Windows'), ('Windows2016', 'Windows(2016)'), ('Other', 'Other')], default='Linux', max_length=128, verbose_name='Platform'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='protocol',
+            field=models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp'), ('telnet', 'telnet (beta)')], default='ssh', max_length=16, verbose_name='Protocol'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='username',
+            field=models.CharField(blank=True, max_length=32, validators=[django.core.validators.RegexValidator('^[0-9a-zA-Z_@\\-\\.]*$', 'Special char not allowed')], verbose_name='Username'),
+        ),
+        migrations.RunPython(
+            code=migrate_win_to_ssh_protocol,
+        ),
+        migrations.AddField(
+            model_name='adminuser',
+            name='org_id',
+            field=models.CharField(blank=True, default=None, max_length=36, null=True),
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='org_id',
+            field=models.CharField(blank=True, default=None, max_length=36, null=True),
+        ),
+        migrations.AddField(
+            model_name='domain',
+            name='org_id',
+            field=models.CharField(blank=True, default=None, max_length=36, null=True),
+        ),
+        migrations.AddField(
+            model_name='gateway',
+            name='org_id',
+            field=models.CharField(blank=True, default=None, max_length=36, null=True),
+        ),
+        migrations.AddField(
+            model_name='label',
+            name='org_id',
+            field=models.CharField(blank=True, default=None, max_length=36, null=True),
+        ),
+        migrations.AddField(
+            model_name='node',
+            name='org_id',
+            field=models.CharField(blank=True, default=None, max_length=36, null=True),
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='org_id',
+            field=models.CharField(blank=True, default=None, max_length=36, null=True),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='name',
+            field=models.CharField(max_length=128, verbose_name='Name'),
+        ),
+        migrations.AlterField(
+            model_name='asset',
+            name='hostname',
+            field=models.CharField(max_length=128, verbose_name='Hostname'),
+        ),
+        migrations.AlterField(
+            model_name='gateway',
+            name='name',
+            field=models.CharField(max_length=128, verbose_name='Name'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='name',
+            field=models.CharField(max_length=128, verbose_name='Name'),
+        ),
+        migrations.AlterUniqueTogether(
+            name='adminuser',
+            unique_together={('name', 'org_id')},
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='cpu_vcpus',
+            field=models.IntegerField(null=True, verbose_name='CPU vcpus'),
+        ),
+        migrations.AlterUniqueTogether(
+            name='asset',
+            unique_together={('org_id', 'hostname')},
+        ),
+        migrations.AlterUniqueTogether(
+            name='gateway',
+            unique_together={('name', 'org_id')},
+        ),
+        migrations.AlterUniqueTogether(
+            name='systemuser',
+            unique_together={('name', 'org_id')},
+        ),
+        migrations.AlterUniqueTogether(
+            name='label',
+            unique_together={('name', 'value', 'org_id')},
+        ),
+    ]

apps/assets/migrations/0020_auto_20180816_1652.py

@@ -0,0 +1,48 @@
+# Generated by Django 2.0.7 on 2018-08-16 08:52
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0019_auto_20180816_1320'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='adminuser',
+            name='org_id',
+            field=models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization'),
+        ),
+        migrations.AlterField(
+            model_name='asset',
+            name='org_id',
+            field=models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization'),
+        ),
+        migrations.AlterField(
+            model_name='domain',
+            name='org_id',
+            field=models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization'),
+        ),
+        migrations.AlterField(
+            model_name='gateway',
+            name='org_id',
+            field=models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization'),
+        ),
+        migrations.AlterField(
+            model_name='label',
+            name='org_id',
+            field=models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization'),
+        ),
+        migrations.AlterField(
+            model_name='node',
+            name='org_id',
+            field=models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='org_id',
+            field=models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization'),
+        ),
+    ]

apps/assets/migrations/0021_auto_20180903_1132.py

@@ -0,0 +1,25 @@
+# Generated by Django 2.1 on 2018-09-03 03:32
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0020_auto_20180816_1652'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='domain',
+            options={'verbose_name': 'Domain'},
+        ),
+        migrations.AlterModelOptions(
+            name='gateway',
+            options={'verbose_name': 'Gateway'},
+        ),
+        migrations.AlterModelOptions(
+            name='node',
+            options={'verbose_name': 'Node'},
+        ),
+    ]

apps/assets/migrations/0022_auto_20181012_1717.py

@@ -0,0 +1,56 @@
+# Generated by Django 2.1.1 on 2018-10-12 09:17
+
+import django.core.validators
+from django.db import migrations, models
+import django.db.models.deletion
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0021_auto_20180903_1132'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='CommandFilter',
+            fields=[
+                ('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(max_length=64, verbose_name='Name')),
+                ('is_active', models.BooleanField(default=True, verbose_name='Is active')),
+                ('comment', models.TextField(blank=True, default='', verbose_name='Comment')),
+                ('date_created', models.DateTimeField(auto_now_add=True)),
+                ('date_updated', models.DateTimeField(auto_now=True)),
+                ('created_by', models.CharField(blank=True, default='', max_length=128, verbose_name='Created by')),
+            ],
+            options={
+                'abstract': False,
+            },
+        ),
+        migrations.CreateModel(
+            name='CommandFilterRule',
+            fields=[
+                ('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('type', models.CharField(choices=[('regex', 'Regex'), ('command', 'Command')], default='command', max_length=16, verbose_name='Type')),
+                ('priority', models.IntegerField(default=50, help_text='1-100, the lower will be match first', validators=[django.core.validators.MinValueValidator(1), django.core.validators.MaxValueValidator(100)], verbose_name='Priority')),
+                ('content', models.TextField(help_text='One line one command', max_length=1024, verbose_name='Content')),
+                ('action', models.IntegerField(choices=[(0, 'Deny'), (1, 'Allow')], default=0, verbose_name='Action')),
+                ('comment', models.CharField(blank=True, default='', max_length=64, verbose_name='Comment')),
+                ('date_created', models.DateTimeField(auto_now_add=True)),
+                ('date_updated', models.DateTimeField(auto_now=True)),
+                ('created_by', models.CharField(blank=True, default='', max_length=128, verbose_name='Created by')),
+                ('filter', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='rules', to='assets.CommandFilter', verbose_name='Filter')),
+            ],
+            options={
+                'ordering': ('priority', 'action'),
+            },
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='cmd_filters',
+            field=models.ManyToManyField(blank=True, related_name='system_users', to='assets.CommandFilter', verbose_name='Command filter'),
+        ),
+    ]

apps/assets/migrations/0023_auto_20181016_1650.py

@@ -0,0 +1,28 @@
+# Generated by Django 2.1.1 on 2018-10-16 08:50
+
+import django.core.validators
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0022_auto_20181012_1717'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='commandfilterrule',
+            options={'ordering': ('-priority', 'action')},
+        ),
+        migrations.AlterField(
+            model_name='commandfilterrule',
+            name='priority',
+            field=models.IntegerField(default=50, help_text='1-100, the higher will be match first', validators=[django.core.validators.MinValueValidator(1), django.core.validators.MaxValueValidator(100)], verbose_name='Priority'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='priority',
+            field=models.IntegerField(default=20, validators=[django.core.validators.MinValueValidator(1), django.core.validators.MaxValueValidator(100)], verbose_name='Priority'),
+        ),
+    ]

apps/assets/migrations/0024_auto_20181219_1614.py

@@ -0,0 +1,23 @@
+# Generated by Django 2.1.4 on 2018-12-19 08:14
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0023_auto_20181016_1650'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='asset',
+            name='protocol',
+            field=models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp'), ('telnet', 'telnet (beta)'), ('vnc', 'vnc')], default='ssh', max_length=128, verbose_name='Protocol'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='protocol',
+            field=models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp'), ('telnet', 'telnet (beta)'), ('vnc', 'vnc')], default='ssh', max_length=16, verbose_name='Protocol'),
+        ),
+    ]

apps/assets/migrations/0025_auto_20190221_1902.py

@@ -0,0 +1,21 @@
+# Generated by Django 2.1.7 on 2019-02-21 11:02
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0024_auto_20181219_1614'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='commandfilter',
+            options={'verbose_name': 'Command filter'},
+        ),
+        migrations.AlterModelOptions(
+            name='commandfilterrule',
+            options={'ordering': ('-priority', 'action'), 'verbose_name': 'Command filter rule'},
+        ),
+    ]

apps/assets/migrations/0026_auto_20190325_2035.py

@@ -0,0 +1,43 @@
+# Generated by Django 2.1.7 on 2019-03-25 12:35
+
+import assets.models.utils
+import django.core.validators
+from django.db import migrations, models
+import django.db.models.deletion
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0025_auto_20190221_1902'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='AuthBook',
+            fields=[
+                ('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(max_length=128, verbose_name='Name')),
+                ('username', models.CharField(blank=True, max_length=32, validators=[django.core.validators.RegexValidator('^[0-9a-zA-Z_@\\-\\.]*$', 'Special char not allowed')], verbose_name='Username')),
+                ('_password', models.CharField(blank=True, max_length=256, null=True, verbose_name='Password')),
+                ('_private_key', models.TextField(blank=True, max_length=4096, null=True, validators=[assets.models.utils.private_key_validator], verbose_name='SSH private key')),
+                ('_public_key', models.TextField(blank=True, max_length=4096, verbose_name='SSH public key')),
+                ('comment', models.TextField(blank=True, verbose_name='Comment')),
+                ('date_created', models.DateTimeField(auto_now_add=True)),
+                ('date_updated', models.DateTimeField(auto_now=True)),
+                ('created_by', models.CharField(max_length=128, null=True, verbose_name='Created by')),
+                ('is_latest', models.BooleanField(default=False, verbose_name='Latest version')),
+                ('version', models.IntegerField(default=1, verbose_name='Version')),
+                ('asset', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='assets.Asset', verbose_name='Asset')),
+            ],
+            options={
+                'verbose_name': 'AuthBook',
+            },
+        ),
+        migrations.AlterModelOptions(
+            name='node',
+            options={'ordering': ['key'], 'verbose_name': 'Node'},
+        ),
+    ]

apps/assets/migrations/0027_auto_20190521_1703.py

@@ -0,0 +1,23 @@
+# Generated by Django 2.1.7 on 2019-05-21 09:03
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0026_auto_20190325_2035'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='asset',
+            name='ip',
+            field=models.CharField(db_index=True, max_length=128, verbose_name='IP'),
+        ),
+        migrations.AlterField(
+            model_name='asset',
+            name='public_ip',
+            field=models.CharField(blank=True, max_length=128, null=True, verbose_name='Public IP'),
+        ),
+    ]

apps/assets/migrations/0028_protocol.py

@@ -0,0 +1,29 @@
+# Generated by Django 2.1.7 on 2019-05-22 02:58
+
+import django.core.validators
+from django.db import migrations, models
+import uuid
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0027_auto_20190521_1703'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Protocol',
+            fields=[
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('name', models.CharField(choices=[('ssh', 'ssh'), ('rdp', 'rdp'), ('telnet', 'telnet (beta)'), ('vnc', 'vnc')], default='ssh', max_length=16, verbose_name='Name')),
+                ('port', models.IntegerField(default=22, validators=[django.core.validators.MaxValueValidator(65535), django.core.validators.MinValueValidator(1)], verbose_name='Port')),
+            ],
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='protocols',
+            field=models.ManyToManyField(to='assets.Protocol',
+                                         verbose_name='Protocol'),
+        ),
+    ]

apps/assets/migrations/0029_auto_20190522_1114.py

@@ -0,0 +1,13 @@
+# Generated by Django 2.1.7 on 2019-05-22 03:14
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0028_protocol'),
+    ]
+
+    operations = [
+    ]

apps/assets/migrations/0030_auto_20190619_1135.py

@@ -0,0 +1,19 @@
+# Generated by Django 2.1.7 on 2019-06-19 03:35
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0029_auto_20190522_1114'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='asset',
+            name='admin_user',
+            field=models.ForeignKey(null=True, on_delete=django.db.models.deletion.PROTECT, related_name='assets', to='assets.AdminUser', verbose_name='Admin user'),
+        ),
+    ]

apps/assets/migrations/0031_auto_20190621_1332.py

@@ -0,0 +1,53 @@
+# Generated by Django 2.1.7 on 2019-06-21 05:32
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0030_auto_20190619_1135'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='adminuser',
+            name='date_created',
+            field=models.DateTimeField(auto_now_add=True, verbose_name='Date created'),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='date_updated',
+            field=models.DateTimeField(auto_now=True, verbose_name='Date updated'),
+        ),
+        migrations.AlterField(
+            model_name='authbook',
+            name='date_created',
+            field=models.DateTimeField(auto_now_add=True, verbose_name='Date created'),
+        ),
+        migrations.AlterField(
+            model_name='authbook',
+            name='date_updated',
+            field=models.DateTimeField(auto_now=True, verbose_name='Date updated'),
+        ),
+        migrations.AlterField(
+            model_name='gateway',
+            name='date_created',
+            field=models.DateTimeField(auto_now_add=True, verbose_name='Date created'),
+        ),
+        migrations.AlterField(
+            model_name='gateway',
+            name='date_updated',
+            field=models.DateTimeField(auto_now=True, verbose_name='Date updated'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='date_created',
+            field=models.DateTimeField(auto_now_add=True, verbose_name='Date created'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='date_updated',
+            field=models.DateTimeField(auto_now=True, verbose_name='Date updated'),
+        ),
+    ]

apps/assets/migrations/0032_auto_20190624_2108.py

@@ -0,0 +1,75 @@
+# Generated by Django 2.1.7 on 2019-06-24 13:08
+
+import assets.models.utils
+import common.fields.model
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0031_auto_20190621_1332'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='adminuser',
+            name='_password',
+            field=common.fields.model.EncryptCharField(blank=True, max_length=256, null=True, verbose_name='Password'),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='_private_key',
+            field=common.fields.model.EncryptTextField(blank=True, null=True, validators=[assets.models.utils.private_key_validator], verbose_name='SSH private key'),
+        ),
+        migrations.AlterField(
+            model_name='adminuser',
+            name='_public_key',
+            field=common.fields.model.EncryptTextField(blank=True, null=True, verbose_name='SSH public key'),
+        ),
+        migrations.AlterField(
+            model_name='authbook',
+            name='_password',
+            field=common.fields.model.EncryptCharField(blank=True, max_length=256, null=True, verbose_name='Password'),
+        ),
+        migrations.AlterField(
+            model_name='authbook',
+            name='_private_key',
+            field=common.fields.model.EncryptTextField(blank=True, null=True, validators=[assets.models.utils.private_key_validator], verbose_name='SSH private key'),
+        ),
+        migrations.AlterField(
+            model_name='authbook',
+            name='_public_key',
+            field=common.fields.model.EncryptTextField(blank=True, null=True, verbose_name='SSH public key'),
+        ),
+        migrations.AlterField(
+            model_name='gateway',
+            name='_password',
+            field=common.fields.model.EncryptCharField(blank=True, max_length=256, null=True, verbose_name='Password'),
+        ),
+        migrations.AlterField(
+            model_name='gateway',
+            name='_private_key',
+            field=common.fields.model.EncryptTextField(blank=True, null=True, validators=[assets.models.utils.private_key_validator], verbose_name='SSH private key'),
+        ),
+        migrations.AlterField(
+            model_name='gateway',
+            name='_public_key',
+            field=common.fields.model.EncryptTextField(blank=True, null=True, verbose_name='SSH public key'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='_password',
+            field=common.fields.model.EncryptCharField(blank=True, max_length=256, null=True, verbose_name='Password'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='_private_key',
+            field=common.fields.model.EncryptTextField(blank=True, null=True, validators=[assets.models.utils.private_key_validator], verbose_name='SSH private key'),
+        ),
+        migrations.AlterField(
+            model_name='systemuser',
+            name='_public_key',
+            field=common.fields.model.EncryptTextField(blank=True, null=True, verbose_name='SSH public key'),
+        ),
+    ]

apps/assets/migrations/0033_auto_20190624_2108.py

@@ -0,0 +1,73 @@
+# Generated by Django 2.1.7 on 2019-06-24 13:08
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0032_auto_20190624_2108'),
+    ]
+
+    operations = [
+        migrations.RenameField(
+            model_name='adminuser',
+            old_name='_private_key',
+            new_name='private_key',
+        ),
+        migrations.RenameField(
+            model_name='adminuser',
+            old_name='_public_key',
+            new_name='public_key',
+        ),
+        migrations.RenameField(
+            model_name='authbook',
+            old_name='_private_key',
+            new_name='private_key',
+        ),
+        migrations.RenameField(
+            model_name='authbook',
+            old_name='_public_key',
+            new_name='public_key',
+        ),
+        migrations.RenameField(
+            model_name='gateway',
+            old_name='_private_key',
+            new_name='private_key',
+        ),
+        migrations.RenameField(
+            model_name='gateway',
+            old_name='_public_key',
+            new_name='public_key',
+        ),
+        migrations.RenameField(
+            model_name='systemuser',
+            old_name='_private_key',
+            new_name='private_key',
+        ),
+        migrations.RenameField(
+            model_name='systemuser',
+            old_name='_public_key',
+            new_name='public_key',
+        ),
+        migrations.RenameField(
+            model_name='adminuser',
+            old_name='_password',
+            new_name='password',
+        ),
+        migrations.RenameField(
+            model_name='authbook',
+            old_name='_password',
+            new_name='password',
+        ),
+        migrations.RenameField(
+            model_name='gateway',
+            old_name='_password',
+            new_name='password',
+        ),
+        migrations.RenameField(
+            model_name='systemuser',
+            old_name='_password',
+            new_name='password',
+        ),
+    ]

apps/assets/migrations/0034_auto_20190705_1348.py

@@ -0,0 +1,39 @@
+# Generated by Django 2.1.7 on 2019-07-05 05:48
+
+from django.db import migrations
+from django.db.models import F
+from django.db.models import CharField, Value as V
+from django.db.models.functions import Concat
+
+
+def migrate_assets_protocol(apps, schema_editor):
+    asset_model = apps.get_model("assets", "Asset")
+    db_alias = schema_editor.connection.alias
+    assets = asset_model.objects.using(db_alias).all().annotate(
+        protocols_new=Concat(
+            'protocol', V('/'), 'port',
+            output_field=CharField(),
+        ),
+    )
+    assets.update(protocols=F('protocols_new'))
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0033_auto_20190624_2108'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='asset',
+            name='protocols',
+        ),
+        migrations.AddField(
+            model_name='asset',
+            name='protocols',
+            field=CharField(blank=True, default='ssh/22', max_length=128, verbose_name='Protocols'),
+        ),
+        migrations.RunPython(migrate_assets_protocol),
+        migrations.DeleteModel(name='Protocol'),
+    ]

apps/assets/models/__init__.py

@@ -1,10 +1,11 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-#
-from .user import AdminUser, SystemUser
+from .asset import *
 from .label import Label
+from .user import *
 from .cluster import *
 from .group import *
+from .domain import *
 from .node import *
-from .asset import *
+from .cmd_filter import *
+from .authbook import *
 from .utils import *
+from .authbook import *

apps/assets/models/asset.py

@@ -4,17 +4,18 @@
 
 import uuid
 import logging
+import random
+from functools import reduce
+from collections import OrderedDict
 
 from django.db import models
 from django.utils.translation import ugettext_lazy as _
-from django.core.cache import cache
 
-from ..const import ASSET_ADMIN_CONN_CACHE_KEY
-from .cluster import Cluster
-from .group import AssetGroup
 from .user import AdminUser, SystemUser
+from .utils import Connectivity
+from orgs.mixins import OrgModelMixin, OrgManager
 
-__all__ = ['Asset']
+__all__ = ['Asset', 'ProtocolsMixin']
 logger = logging.getLogger(__name__)
 
 
@@ -31,12 +32,71 @@ def default_cluster():
 def default_node():
     try:
         from .node import Node
-        return Node.root()
+        root = Node.root()
+        return root
     except:
         return None
 
 
-class Asset(models.Model):
+class AssetQuerySet(models.QuerySet):
+    def active(self):
+        return self.filter(is_active=True)
+
+    def valid(self):
+        return self.active()
+
+    def has_protocol(self, name):
+        return self.filter(protocols__contains=name)
+
+
+class ProtocolsMixin:
+    protocols = ''
+    PROTOCOL_SSH = 'ssh'
+    PROTOCOL_RDP = 'rdp'
+    PROTOCOL_TELNET = 'telnet'
+    PROTOCOL_VNC = 'vnc'
+    PROTOCOL_CHOICES = (
+        (PROTOCOL_SSH, 'ssh'),
+        (PROTOCOL_RDP, 'rdp'),
+        (PROTOCOL_TELNET, 'telnet (beta)'),
+        (PROTOCOL_VNC, 'vnc'),
+    )
+
+    @property
+    def protocols_as_list(self):
+        if not self.protocols:
+            return []
+        return self.protocols.split(' ')
+
+    @property
+    def protocols_as_dict(self):
+        d = OrderedDict()
+        protocols = self.protocols_as_list
+        for i in protocols:
+            if '/' not in i:
+                continue
+            name, port = i.split('/')[:2]
+            if not all([name, port]):
+                continue
+            d[name] = int(port)
+        return d
+
+    @property
+    def protocols_as_json(self):
+        return [
+            {"name": name, "port": port}
+            for name, port in self.protocols_as_dict.items()
+        ]
+
+    def has_protocol(self, name):
+        return name in self.protocols_as_dict
+
+    @property
+    def ssh_port(self):
+        return self.protocols_as_dict.get("ssh", 22)
+
+
+class Asset(ProtocolsMixin, OrgModelMixin):
     # Important
     PLATFORM_CHOICES = (
         ('Linux', 'Linux'),
@@ -44,20 +104,29 @@ class Asset(models.Model):
         ('MacOS', 'MacOS'),
         ('BSD', 'BSD'),
         ('Windows', 'Windows'),
+        ('Windows2016', 'Windows(2016)'),
         ('Other', 'Other'),
     )
+
     id = models.UUIDField(default=uuid.uuid4, primary_key=True)
-    ip = models.GenericIPAddressField(max_length=32, verbose_name=_('IP'), db_index=True)
-    hostname = models.CharField(max_length=128, unique=True, verbose_name=_('Hostname'))
+    ip = models.CharField(max_length=128, verbose_name=_('IP'), db_index=True)
+    hostname = models.CharField(max_length=128, verbose_name=_('Hostname'))
+    protocol = models.CharField(max_length=128, default=ProtocolsMixin.PROTOCOL_SSH,
+                                choices=ProtocolsMixin.PROTOCOL_CHOICES,
+                                verbose_name=_('Protocol'))
     port = models.IntegerField(default=22, verbose_name=_('Port'))
+
+    protocols = models.CharField(max_length=128, default='ssh/22', blank=True, verbose_name=_("Protocols"))
+    platform = models.CharField(max_length=128, choices=PLATFORM_CHOICES, default='Linux', verbose_name=_('Platform'))
+    domain = models.ForeignKey("assets.Domain", null=True, blank=True, related_name='assets', verbose_name=_("Domain"), on_delete=models.SET_NULL)
     nodes = models.ManyToManyField('assets.Node', default=default_node, related_name='assets', verbose_name=_("Nodes"))
     is_active = models.BooleanField(default=True, verbose_name=_('Is active'))
 
     # Auth
-    admin_user = models.ForeignKey('assets.AdminUser', on_delete=models.PROTECT, null=True, verbose_name=_("Admin user"))
+    admin_user = models.ForeignKey('assets.AdminUser', on_delete=models.PROTECT, null=True, verbose_name=_("Admin user"), related_name='assets')
 
     # Some information
-    public_ip = models.GenericIPAddressField(max_length=32, blank=True, null=True, verbose_name=_('Public IP'))
+    public_ip = models.CharField(max_length=128, blank=True, null=True, verbose_name=_('Public IP'))
     number = models.CharField(max_length=32, null=True, blank=True, verbose_name=_('Asset number'))
 
     # Collect
@@ -68,11 +137,11 @@ class Asset(models.Model):
     cpu_model = models.CharField(max_length=64, null=True, blank=True, verbose_name=_('CPU model'))
     cpu_count = models.IntegerField(null=True, verbose_name=_('CPU count'))
     cpu_cores = models.IntegerField(null=True, verbose_name=_('CPU cores'))
+    cpu_vcpus = models.IntegerField(null=True, verbose_name=_('CPU vcpus'))
     memory = models.CharField(max_length=64, null=True, blank=True, verbose_name=_('Memory'))
     disk_total = models.CharField(max_length=1024, null=True, blank=True, verbose_name=_('Disk total'))
     disk_info = models.CharField(max_length=1024, null=True, blank=True, verbose_name=_('Disk info'))
 
-    platform = models.CharField(max_length=128, choices=PLATFORM_CHOICES, default='Linux', verbose_name=_('Platform'))
     os = models.CharField(max_length=128, null=True, blank=True, verbose_name=_('OS'))
     os_version = models.CharField(max_length=16, null=True, blank=True, verbose_name=_('OS version'))
     os_arch = models.CharField(max_length=16, blank=True, null=True, verbose_name=_('OS arch'))
@@ -83,94 +152,168 @@ class Asset(models.Model):
     date_created = models.DateTimeField(auto_now_add=True, null=True, blank=True, verbose_name=_('Date created'))
     comment = models.TextField(max_length=128, default='', blank=True, verbose_name=_('Comment'))
 
+    objects = OrgManager.from_queryset(AssetQuerySet)()
+    _connectivity = None
+
     def __str__(self):
-        return self.hostname
+        return '{0.hostname}({0.ip})'.format(self)
 
     @property
     def is_valid(self):
         warning = ''
         if not self.is_active:
             warning += ' inactive'
-        else:
-            return True, ''
-        return False, warning
+        if warning:
+            return False, warning
+        return True, warning
 
-    def is_unixlike(self):
-        if self.platform not in ("Windows", "Other"):
+    def is_windows(self):
+        if self.platform in ("Windows", "Windows2016"):
             return True
         else:
             return False
 
+    def is_unixlike(self):
+        if self.platform not in ("Windows", "Windows2016", "Other"):
+            return True
+        else:
+            return False
+
+    def is_support_ansible(self):
+        return self.has_protocol('ssh') and self.platform not in ("Other",)
+
+    def get_nodes(self):
+        from .node import Node
+        nodes = self.nodes.all() or [Node.root()]
+        return nodes
+
+    def get_all_nodes(self, flat=False):
+        nodes = []
+        for node in self.get_nodes():
+            _nodes = node.get_ancestor(with_self=True)
+            nodes.append(_nodes)
+        if flat:
+            nodes = list(reduce(lambda x, y: set(x) | set(y), nodes))
+        return nodes
+
+    @property
+    def cpu_info(self):
+        info = ""
+        if self.cpu_model:
+            info += self.cpu_model
+        if self.cpu_count and self.cpu_cores:
+            info += "{}*{}".format(self.cpu_count, self.cpu_cores)
+        return info
+
     @property
     def hardware_info(self):
         if self.cpu_count:
             return '{} Core {} {}'.format(
-                self.cpu_count * self.cpu_cores,
+                self.cpu_vcpus or self.cpu_count * self.cpu_cores,
                 self.memory, self.disk_total
             )
         else:
             return ''
 
     @property
-    def is_connective(self):
-        if not self.is_unixlike():
-            return True
-        val = cache.get(ASSET_ADMIN_CONN_CACHE_KEY.format(self.hostname))
-        if val == 1:
-            return True
-        else:
-            return False
+    def connectivity(self):
+        if self._connectivity:
+            return self._connectivity
+        if not self.admin_user:
+            return Connectivity.unknown()
+        connectivity = self.admin_user.get_asset_connectivity(self)
+        return connectivity
 
-    def to_json(self):
-        return {
-            'id': self.id,
-            'hostname': self.hostname,
-            'ip': self.ip,
-            'port': self.port,
+    @connectivity.setter
+    def connectivity(self, value):
+        if not self.admin_user:
+            return
+        self.admin_user.set_asset_connectivity(self, value)
+
+    def get_auth_info(self):
+        if not self.admin_user:
+            return {}
+
+        self.admin_user.load_specific_asset_auth(self)
+        info = {
+            'username': self.admin_user.username,
+            'password': self.admin_user.password,
+            'private_key': self.admin_user.private_key_file,
         }
+        return info
 
-    def _to_secret_json(self):
-        """
-        Ansible use it create inventory, First using asset user,
-        otherwise using cluster admin user
+    def as_node(self):
+        from .node import Node
+        fake_node = Node()
+        fake_node.id = self.id
+        fake_node.key = self.id
+        fake_node.value = self.hostname
+        fake_node.asset = self
+        fake_node.is_node = False
+        return fake_node
 
-        Todo: May be move to ops implements it
-        """
-        data = self.to_json()
-        if self.admin_user:
-            admin_user = self.admin_user
-            data.update({
-                'username': admin_user.username,
-                'password': admin_user.password,
-                'private_key': admin_user.private_key_file,
-                'become': admin_user.become_info,
-                'groups': [node.value for node in self.nodes.all()],
-            })
-        return data
+    def as_tree_node(self, parent_node):
+        from common.tree import TreeNode
+        icon_skin = 'file'
+        if self.platform.lower() == 'windows':
+            icon_skin = 'windows'
+        elif self.platform.lower() == 'linux':
+            icon_skin = 'linux'
+        data = {
+            'id': str(self.id),
+            'name': self.hostname,
+            'title': self.ip,
+            'pId': parent_node.key,
+            'isParent': False,
+            'open': False,
+            'iconSkin': icon_skin,
+            'meta': {
+                'type': 'asset',
+                'asset': {
+                    'id': self.id,
+                    'hostname': self.hostname,
+                    'ip': self.ip,
+                    'protocols': self.protocols_as_list,
+                    'platform': self.platform,
+                }
+            }
+        }
+        tree_node = TreeNode(**data)
+        return tree_node
 
     class Meta:
-        unique_together = ('ip', 'port')
+        unique_together = [('org_id', 'hostname')]
         verbose_name = _("Asset")
 
     @classmethod
     def generate_fake(cls, count=100):
         from random import seed, choice
-        import forgery_py
         from django.db import IntegrityError
+        from .node import Node
+        from orgs.utils import get_current_org
+        from orgs.models import Organization
+        org = get_current_org()
+        if not org or not org.is_real():
+            Organization.default().change_to()
 
+        nodes = list(Node.objects.all())
         seed()
         for i in range(count):
-            asset = cls(ip='%s.%s.%s.%s' % (i, i, i, i),
-                        hostname=forgery_py.internet.user_name(True),
+            ip = [str(i) for i in random.sample(range(255), 4)]
+            asset = cls(ip='.'.join(ip),
+                        hostname='.'.join(ip),
                         admin_user=choice(AdminUser.objects.all()),
-                        port=22,
                         created_by='Fake')
             try:
                 asset.save()
+                asset.protocols = 'ssh/22'
+                if nodes and len(nodes) > 3:
+                    _nodes = random.sample(nodes, 3)
+                else:
+                    _nodes = [Node.default_node()]
+                asset.nodes.set(_nodes)
                 asset.system_users = [choice(SystemUser.objects.all()) for i in range(3)]
-                asset.groups = [choice(AssetGroup.objects.all()) for i in range(3)]
                 logger.debug('Generate fake asset : %s' % asset.ip)
             except IntegrityError:
                 print('Error continue')
                 continue
-

apps/assets/models/authbook.py

@@ -0,0 +1,90 @@
+# -*- coding: utf-8 -*-
+#
+
+from django.db import models
+from django.utils.translation import ugettext_lazy as _
+
+from orgs.mixins import OrgManager
+from .base import AssetUser
+
+__all__ = ['AuthBook']
+
+
+class AuthBookQuerySet(models.QuerySet):
+
+    def latest_version(self):
+        return self.filter(is_latest=True)
+
+
+class AuthBookManager(OrgManager):
+    pass
+
+
+class AuthBook(AssetUser):
+    asset = models.ForeignKey('assets.Asset', on_delete=models.CASCADE, verbose_name=_('Asset'))
+    is_latest = models.BooleanField(default=False, verbose_name=_('Latest version'))
+    version = models.IntegerField(default=1, verbose_name=_('Version'))
+
+    objects = AuthBookManager.from_queryset(AuthBookQuerySet)()
+    backend = "db"
+    # 用于system user和admin_user的动态设置
+    _connectivity = None
+    CONN_CACHE_KEY = "ASSET_USER_CONN_{}"
+
+    class Meta:
+        verbose_name = _('AuthBook')
+
+    def _set_latest(self):
+        self._remove_pre_obj_latest()
+        self.is_latest = True
+        self.save()
+
+    def _get_pre_obj(self):
+        pre_obj = self.__class__.objects.filter(
+            username=self.username, asset=self.asset
+        ).latest_version().first()
+        return pre_obj
+
+    def _remove_pre_obj_latest(self):
+        pre_obj = self._get_pre_obj()
+        if pre_obj:
+            pre_obj.is_latest = False
+            pre_obj.save()
+
+    def _set_version(self):
+        pre_obj = self._get_pre_obj()
+        if pre_obj:
+            self.version = pre_obj.version + 1
+        else:
+            self.version = 1
+        self.save()
+
+    def set_version_and_latest(self):
+        self._set_version()
+        self._set_latest()
+
+    def get_related_assets(self):
+        return [self.asset]
+
+    def generate_id_with_asset(self, asset):
+        return self.id
+
+    @property
+    def connectivity(self):
+        return self.get_asset_connectivity(self.asset)
+
+    @property
+    def keyword(self):
+        return '{}_#_{}'.format(self.username, str(self.asset.id))
+
+    @property
+    def hostname(self):
+        return self.asset.hostname
+
+    @property
+    def ip(self):
+        return self.asset.ip
+
+    def __str__(self):
+        return '{}@{}'.format(self.username, self.asset)
+

apps/assets/models/base.py

@@ -0,0 +1,246 @@
+# -*- coding: utf-8 -*-
+#
+import os
+import uuid
+from hashlib import md5
+
+import sshpubkeys
+from django.core.cache import cache
+from django.db import models
+from django.utils.translation import ugettext_lazy as _
+from django.conf import settings
+
+from common.utils import (
+    get_signer, ssh_key_string_to_obj, ssh_key_gen, get_logger
+)
+from common.validators import alphanumeric
+from common import fields
+from orgs.mixins import OrgModelMixin
+from .utils import private_key_validator, Connectivity
+
+signer = get_signer()
+
+logger = get_logger(__file__)
+
+
+class AssetUser(OrgModelMixin):
+    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
+    name = models.CharField(max_length=128, verbose_name=_('Name'))
+    username = models.CharField(max_length=32, blank=True, verbose_name=_('Username'), validators=[alphanumeric])
+    password = fields.EncryptCharField(max_length=256, blank=True, null=True, verbose_name=_('Password'))
+    private_key = fields.EncryptTextField(blank=True, null=True, verbose_name=_('SSH private key'), validators=[private_key_validator, ])
+    public_key = fields.EncryptTextField(blank=True, null=True, verbose_name=_('SSH public key'))
+    comment = models.TextField(blank=True, verbose_name=_('Comment'))
+    date_created = models.DateTimeField(auto_now_add=True, verbose_name=_("Date created"))
+    date_updated = models.DateTimeField(auto_now=True, verbose_name=_("Date updated"))
+    created_by = models.CharField(max_length=128, null=True, verbose_name=_('Created by'))
+
+    CONNECTIVITY_ASSET_CACHE_KEY = "ASSET_USER_{}_{}_ASSET_CONNECTIVITY"
+    CONNECTIVITY_AMOUNT_CACHE_KEY = "ASSET_USER_{}_{}_CONNECTIVITY_AMOUNT"
+    ASSETS_AMOUNT_CACHE_KEY = "ASSET_USER_{}_ASSETS_AMOUNT"
+    ASSET_USER_CACHE_TIME = 3600 * 24
+
+    _prefer = "system_user"
+
+    @property
+    def private_key_obj(self):
+        if self.private_key:
+            return ssh_key_string_to_obj(self.private_key, password=self.password)
+        else:
+            return None
+
+    @property
+    def private_key_file(self):
+        if not self.private_key_obj:
+            return None
+        project_dir = settings.PROJECT_DIR
+        tmp_dir = os.path.join(project_dir, 'tmp')
+        key_name = '.' + md5(self.private_key.encode('utf-8')).hexdigest()
+        key_path = os.path.join(tmp_dir, key_name)
+        if not os.path.exists(key_path):
+            self.private_key_obj.write_private_key_file(key_path)
+            os.chmod(key_path, 0o400)
+        return key_path
+
+    @property
+    def public_key_obj(self):
+        if self.public_key:
+            try:
+                return sshpubkeys.SSHKey(self.public_key)
+            except TabError:
+                pass
+        return None
+
+    @property
+    def part_id(self):
+        i = '-'.join(str(self.id).split('-')[:3])
+        return i
+
+    def get_related_assets(self):
+        assets = self.assets.all()
+        return assets
+
+    def set_auth(self, password=None, private_key=None, public_key=None):
+        update_fields = []
+        if password:
+            self.password = password
+            update_fields.append('password')
+        if private_key:
+            self.private_key = private_key
+            update_fields.append('private_key')
+        if public_key:
+            self.public_key = public_key
+            update_fields.append('public_key')
+
+        if update_fields:
+            self.save(update_fields=update_fields)
+
+    def set_connectivity(self, summary):
+        unreachable = summary.get('dark', {}).keys()
+        reachable = summary.get('contacted', {}).keys()
+
+        assets = self.get_related_assets()
+        if not isinstance(assets, list):
+            assets = assets.only('id', 'hostname', 'admin_user__id')
+        for asset in assets:
+            if asset.hostname in unreachable:
+                self.set_asset_connectivity(asset, Connectivity.unreachable())
+            elif asset.hostname in reachable:
+                self.set_asset_connectivity(asset, Connectivity.reachable())
+            else:
+                self.set_asset_connectivity(asset, Connectivity.unknown())
+        cache_key = self.CONNECTIVITY_AMOUNT_CACHE_KEY.format(self.username, self.part_id)
+        cache.delete(cache_key)
+
+    @property
+    def connectivity(self):
+        assets = self.get_related_assets()
+        if not isinstance(assets, list):
+            assets = assets.only('id', 'hostname', 'admin_user__id')
+        data = {
+            'unreachable': [],
+            'reachable': [],
+            'unknown': [],
+        }
+        for asset in assets:
+            connectivity = self.get_asset_connectivity(asset)
+            if connectivity.is_reachable():
+                data["reachable"].append(asset.hostname)
+            elif connectivity.is_unreachable():
+                data["unreachable"].append(asset.hostname)
+            else:
+                data["unknown"].append(asset.hostname)
+        return data
+
+    @property
+    def connectivity_amount(self):
+        cache_key = self.CONNECTIVITY_AMOUNT_CACHE_KEY.format(self.username, self.part_id)
+        amount = cache.get(cache_key)
+        if not amount:
+            amount = {k: len(v) for k, v in self.connectivity.items()}
+            cache.set(cache_key, amount, self.ASSET_USER_CACHE_TIME)
+        return amount
+
+    @property
+    def assets_amount(self):
+        cache_key = self.ASSETS_AMOUNT_CACHE_KEY.format(self.id)
+        cached = cache.get(cache_key)
+        if not cached:
+            cached = self.get_related_assets().count()
+            cache.set(cache_key, cached, self.ASSET_USER_CACHE_TIME)
+        return cached
+
+    def expire_assets_amount(self):
+        cache_key = self.ASSETS_AMOUNT_CACHE_KEY.format(self.id)
+        cache.delete(cache_key)
+
+    def get_asset_connectivity(self, asset):
+        key = self.get_asset_connectivity_key(asset)
+        return Connectivity.get(key)
+
+    def get_asset_connectivity_key(self, asset):
+        return self.CONNECTIVITY_ASSET_CACHE_KEY.format(self.username, asset.id)
+
+    def set_asset_connectivity(self, asset, c):
+        key = self.get_asset_connectivity_key(asset)
+        Connectivity.set(key, c)
+        # 当为某个系统用户或管理用户设置的的时候，失效掉他们的连接数量
+        amount_key = self.CONNECTIVITY_AMOUNT_CACHE_KEY.format(self.username, '*')
+        cache.delete_pattern(amount_key)
+
+    def get_asset_user(self, asset):
+        from ..backends import AssetUserManager
+        try:
+            manager = AssetUserManager().prefer(self._prefer)
+            other = manager.get(username=self.username, asset=asset, prefer_id=self.id)
+            return other
+        except Exception as e:
+            logger.error(e, exc_info=True)
+            return None
+
+    def load_specific_asset_auth(self, asset):
+        instance = self.get_asset_user(asset)
+        if instance:
+            self._merge_auth(instance)
+
+    def _merge_auth(self, other):
+        if other.password:
+            self.password = other.password
+        if other.public_key:
+            self.public_key = other.public_key
+        if other.private_key:
+            self.private_key = other.private_key
+
+    def clear_auth(self):
+        self.password = ''
+        self.private_key = ''
+        self.public_key = ''
+        self.save()
+
+    def auto_gen_auth(self):
+        password = str(uuid.uuid4())
+        private_key, public_key = ssh_key_gen(
+            username=self.username
+        )
+        self.set_auth(
+            password=password, private_key=private_key,
+            public_key=public_key
+        )
+
+    def auto_gen_auth_password(self):
+        password = str(uuid.uuid4())
+        self.set_auth(password=password)
+
+    def _to_secret_json(self):
+        """Push system user use it"""
+        return {
+            'name': self.name,
+            'username': self.username,
+            'password': self.password,
+            'public_key': self.public_key,
+            'private_key': self.private_key_file,
+        }
+
+    def generate_id_with_asset(self, asset):
+        user_id = [self.part_id]
+        asset_id = str(asset.id).split('-')[3:]
+        ids = user_id + asset_id
+        return '-'.join(ids)
+
+    def construct_to_authbook(self, asset):
+        from . import AuthBook
+        fields = [
+            'name', 'username', 'comment', 'org_id',
+            'password', 'private_key', 'public_key',
+            'date_created', 'date_updated', 'created_by'
+        ]
+        i = self.generate_id_with_asset(asset)
+        obj = AuthBook(id=i, asset=asset, version=0, is_latest=True)
+        for field in fields:
+            value = getattr(self, field)
+            setattr(obj, field, value)
+        return obj
+
+    class Meta:
+        abstract = True
+

apps/assets/models/cluster.py

@@ -52,7 +52,8 @@ class Cluster(models.Model):
                           contact=forgery_py.name.full_name(),
                           phone=forgery_py.address.phone(),
                           address=forgery_py.address.city() + forgery_py.address.street_address(),
-                          operator=choice(['北京联通', '北京电信', 'BGP全网通']),
+                          # operator=choice(['北京联通', '北京电信', 'BGP全网通']),
+                          operator=choice([_('Beijing unicom'), _('Beijing telecom'), _('BGP full netcom')]),
                           comment=forgery_py.lorem_ipsum.sentence(),
                           created_by='Fake')
             try:

apps/assets/models/cmd_filter.py

@@ -0,0 +1,91 @@
+# -*- coding: utf-8 -*-
+#
+import uuid
+import re
+
+from django.db import models
+from django.core.validators import MinValueValidator, MaxValueValidator
+from django.utils.translation import ugettext_lazy as _
+
+from orgs.mixins import OrgModelMixin
+
+
+__all__ = [
+    'CommandFilter', 'CommandFilterRule'
+]
+
+
+class CommandFilter(OrgModelMixin):
+    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
+    name = models.CharField(max_length=64, verbose_name=_("Name"))
+    is_active = models.BooleanField(default=True, verbose_name=_('Is active'))
+    comment = models.TextField(blank=True, default='', verbose_name=_("Comment"))
+    date_created = models.DateTimeField(auto_now_add=True)
+    date_updated = models.DateTimeField(auto_now=True)
+    created_by = models.CharField(max_length=128, blank=True, default='', verbose_name=_('Created by'))
+
+    def __str__(self):
+        return self.name
+
+    class Meta:
+        verbose_name = _("Command filter")
+
+
+class CommandFilterRule(OrgModelMixin):
+    TYPE_REGEX = 'regex'
+    TYPE_COMMAND = 'command'
+    TYPE_CHOICES = (
+        (TYPE_REGEX, _('Regex')),
+        (TYPE_COMMAND, _('Command')),
+    )
+
+    ACTION_DENY, ACTION_ALLOW, ACTION_UNKNOWN = range(3)
+    ACTION_CHOICES = (
+        (ACTION_DENY, _('Deny')),
+        (ACTION_ALLOW, _('Allow')),
+    )
+
+    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
+    filter = models.ForeignKey('CommandFilter', on_delete=models.CASCADE, verbose_name=_("Filter"), related_name='rules')
+    type = models.CharField(max_length=16, default=TYPE_COMMAND, choices=TYPE_CHOICES, verbose_name=_("Type"))
+    priority = models.IntegerField(default=50, verbose_name=_("Priority"), help_text=_("1-100, the higher will be match first"),
+                                   validators=[MinValueValidator(1), MaxValueValidator(100)])
+    content = models.TextField(max_length=1024, verbose_name=_("Content"), help_text=_("One line one command"))
+    action = models.IntegerField(default=ACTION_DENY, choices=ACTION_CHOICES, verbose_name=_("Action"))
+    comment = models.CharField(max_length=64, blank=True, default='', verbose_name=_("Comment"))
+    date_created = models.DateTimeField(auto_now_add=True)
+    date_updated = models.DateTimeField(auto_now=True)
+    created_by = models.CharField(max_length=128, blank=True, default='', verbose_name=_('Created by'))
+
+    __pattern = None
+
+    class Meta:
+        ordering = ('-priority', 'action')
+        verbose_name = _("Command filter rule")
+
+    @property
+    def _pattern(self):
+        if self.__pattern:
+            return self.__pattern
+        if self.type == 'command':
+            regex = []
+            for cmd in self.content.split('\r\n'):
+                cmd = cmd.replace(' ', '\s+')
+                regex.append(r'\b{0}\b'.format(cmd))
+            self.__pattern = re.compile(r'{}'.format('|'.join(regex)))
+        else:
+            self.__pattern = re.compile(r'{0}'.format(self.content))
+        return self.__pattern
+
+    def match(self, data):
+        found = self._pattern.search(data)
+        if not found:
+            return self.ACTION_UNKNOWN, ''
+
+        if self.action == self.ACTION_ALLOW:
+            return self.ACTION_ALLOW, found.group()
+        else:
+            return self.ACTION_DENY, found.group()
+
+    def __str__(self):
+        return '{} % {}'.format(self.type, self.content)

apps/assets/models/domain.py

@@ -0,0 +1,96 @@
+# -*- coding: utf-8 -*-
+#
+
+import uuid
+import random
+
+import paramiko
+
+from django.db import models
+from django.utils.translation import ugettext_lazy as _
+
+from orgs.mixins import OrgModelMixin
+from .base import AssetUser
+
+__all__ = ['Domain', 'Gateway']
+
+
+class Domain(OrgModelMixin):
+    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
+    name = models.CharField(max_length=128, unique=True, verbose_name=_('Name'))
+    comment = models.TextField(blank=True, verbose_name=_('Comment'))
+    date_created = models.DateTimeField(auto_now_add=True, null=True,
+                                        verbose_name=_('Date created'))
+
+    class Meta:
+        verbose_name = _("Domain")
+
+    def __str__(self):
+        return self.name
+
+    def has_gateway(self):
+        return self.gateway_set.filter(is_active=True).exists()
+
+    @property
+    def gateways(self):
+        return self.gateway_set.filter(is_active=True)
+
+    def random_gateway(self):
+        return random.choice(self.gateways)
+
+
+class Gateway(AssetUser):
+    PROTOCOL_SSH = 'ssh'
+    PROTOCOL_RDP = 'rdp'
+    PROTOCOL_CHOICES = (
+        (PROTOCOL_SSH, 'ssh'),
+        (PROTOCOL_RDP, 'rdp'),
+    )
+    ip = models.GenericIPAddressField(max_length=32, verbose_name=_('IP'), db_index=True)
+    port = models.IntegerField(default=22, verbose_name=_('Port'))
+    protocol = models.CharField(choices=PROTOCOL_CHOICES, max_length=16, default=PROTOCOL_SSH, verbose_name=_("Protocol"))
+    domain = models.ForeignKey(Domain, on_delete=models.CASCADE, verbose_name=_("Domain"))
+    comment = models.CharField(max_length=128, blank=True, null=True, verbose_name=_("Comment"))
+    is_active = models.BooleanField(default=True, verbose_name=_("Is active"))
+
+    def __str__(self):
+        return self.name
+
+    class Meta:
+        unique_together = [('name', 'org_id')]
+        verbose_name = _("Gateway")
+
+    def test_connective(self, local_port=None):
+        if local_port is None:
+            local_port = self.port
+        client = paramiko.SSHClient()
+        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+        proxy = paramiko.SSHClient()
+        proxy.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+
+        try:
+            proxy.connect(self.ip, port=self.port,
+                          username=self.username,
+                          password=self.password,
+                          pkey=self.private_key_obj)
+        except(paramiko.AuthenticationException,
+               paramiko.BadAuthenticationType,
+               paramiko.SSHException) as e:
+            return False, str(e)
+
+        try:
+            sock = proxy.get_transport().open_channel(
+                'direct-tcpip', ('127.0.0.1', local_port), ('127.0.0.1', 0)
+            )
+            client.connect("127.0.0.1", port=local_port,
+                           username=self.username,
+                           password=self.password,
+                           key_filename=self.private_key_file,
+                           sock=sock,
+                           timeout=5)
+        except (paramiko.SSHException, paramiko.ssh_exception.SSHException,
+                paramiko.AuthenticationException, TimeoutError) as e:
+            return False, str(e)
+        finally:
+            client.close()
+        return True, None

apps/assets/models/label.py

@@ -4,9 +4,10 @@
 import uuid
 from django.db import models
 from django.utils.translation import ugettext_lazy as _
+from orgs.mixins import OrgModelMixin
 
 
-class Label(models.Model):
+class Label(OrgModelMixin):
     SYSTEM_CATEGORY = "S"
     USER_CATEGORY = "U"
     CATEGORY_CHOICES = (
@@ -16,7 +17,8 @@ class Label(models.Model):
     id = models.UUIDField(default=uuid.uuid4, primary_key=True)
     name = models.CharField(max_length=128, verbose_name=_("Name"))
     value = models.CharField(max_length=128, verbose_name=_("Value"))
-    category = models.CharField(max_length=128, choices=CATEGORY_CHOICES, default=USER_CATEGORY, verbose_name=_("Category"))
+    category = models.CharField(max_length=128, choices=CATEGORY_CHOICES,
+                                default=USER_CATEGORY, verbose_name=_("Category"))
     is_active = models.BooleanField(default=True, verbose_name=_("Is active"))
     comment = models.TextField(blank=True, null=True, verbose_name=_("Comment"))
     date_created = models.DateTimeField(
@@ -34,4 +36,4 @@ class Label(models.Model):
 
     class Meta:
         db_table = "assets_label"
-        unique_together = ('name', 'value')
+        unique_together = [('name', 'value', 'org_id')]

apps/assets/models/node.py

@@ -1,35 +1,294 @@
 # -*- coding: utf-8 -*-
 #
 import uuid
+import re
 
-from django.db import models
+from django.db import models, transaction
+from django.db.models import Q
 from django.utils.translation import ugettext_lazy as _
+from django.utils.translation import ugettext
+from django.core.cache import cache
 
+from orgs.mixins import OrgModelMixin, OrgManager
+from orgs.utils import set_current_org, get_current_org
+from orgs.models import Organization
 
 __all__ = ['Node']
 
 
-class Node(models.Model):
+class NodeQuerySet(models.QuerySet):
+    def delete(self):
+        raise PermissionError("Bulk delete node deny")
+
+
+class FamilyMixin:
+    _parents = None
+    _children = None
+    _all_children = None
+    is_node = True
+
+    @property
+    def children(self):
+        if self._children:
+            return self._children
+        pattern = r'^{0}:[0-9]+$'.format(self.key)
+        return Node.objects.filter(key__regex=pattern)
+
+    @children.setter
+    def children(self, value):
+        self._children = value
+
+    @property
+    def all_children(self):
+        if self._all_children:
+            return self._all_children
+        pattern = r'^{0}:'.format(self.key)
+        return Node.objects.filter(
+            key__regex=pattern
+        )
+
+    def get_children(self, with_self=False):
+        children = list(self.children)
+        if with_self:
+            children.append(self)
+        return children
+
+    def get_all_children(self, with_self=False):
+        children = self.all_children
+        if with_self:
+            children = list(children)
+            children.append(self)
+        return children
+
+    @property
+    def parents(self):
+        if self._parents:
+            return self._parents
+        ancestor_keys = self.get_ancestor_keys()
+        ancestor = Node.objects.filter(
+            key__in=ancestor_keys
+        ).order_by('key')
+        return ancestor
+
+    @parents.setter
+    def parents(self, value):
+        self._parents = value
+
+    def get_ancestor(self, with_self=False):
+        parents = self.parents
+        if with_self:
+            parents = list(parents)
+            parents.append(self)
+        return parents
+
+    @property
+    def parent(self):
+        if self._parents:
+            return self._parents[0]
+        if self.is_root():
+            return self
+        try:
+            parent = Node.objects.get(key=self.parent_key)
+            return parent
+        except Node.DoesNotExist:
+            return Node.root()
+
+    @parent.setter
+    def parent(self, parent):
+        if not self.is_node:
+            self.key = parent.key + ':fake'
+            return
+        children = self.get_all_children()
+        old_key = self.key
+        with transaction.atomic():
+            self.key = parent.get_next_child_key()
+            for child in children:
+                child.key = child.key.replace(old_key, self.key, 1)
+                child.save()
+            self.save()
+
+    def get_sibling(self, with_self=False):
+        key = ':'.join(self.key.split(':')[:-1])
+        pattern = r'^{}:[0-9]+$'.format(key)
+        sibling = Node.objects.filter(
+            key__regex=pattern.format(self.key)
+        )
+        if not with_self:
+            sibling = sibling.exclude(key=self.key)
+        return sibling
+
+    def get_family(self):
+        ancestor = self.get_ancestor()
+        children = self.get_all_children()
+        return [*tuple(ancestor), self, *tuple(children)]
+
+    def get_ancestor_keys(self, with_self=False):
+        parent_keys = []
+        key_list = self.key.split(":")
+        if not with_self:
+            key_list.pop()
+        for i in range(len(key_list)):
+            parent_keys.append(":".join(key_list))
+            key_list.pop()
+        return parent_keys
+
+    def is_children(self, other):
+        pattern = re.compile(r'^{0}:[0-9]+$'.format(self.key))
+        return pattern.match(other.key)
+
+    def is_parent(self, other):
+        pattern = re.compile(r'^{0}:[0-9]+$'.format(other.key))
+        return pattern.match(self.key)
+
+    @property
+    def parent_key(self):
+        parent_key = ":".join(self.key.split(":")[:-1])
+        return parent_key
+
+    @property
+    def parents_keys(self, with_self=False):
+        keys = []
+        key_list = self.key.split(":")
+        if not with_self:
+            key_list.pop()
+        for i in range(len(key_list)):
+            keys.append(':'.join(key_list))
+            key_list.pop()
+        return keys
+
+
+class FullValueMixin:
+    _full_value_cache_key = '_NODE_VALUE_{}'
+    _full_value = ''
+    key = ''
+
+    @property
+    def full_value(self):
+        if self._full_value:
+            return self._full_value
+        key = self._full_value_cache_key.format(self.key)
+        cached = cache.get(key)
+        if cached:
+            return cached
+        if self.is_root():
+            return self.value
+        parent_full_value = self.parent.full_value
+        value = parent_full_value + ' / ' + self.value
+        self.full_value = value
+        return value
+
+    @full_value.setter
+    def full_value(self, value):
+        self._full_value = value
+        key = self._full_value_cache_key.format(self.key)
+        cache.set(key, value, 3600*24)
+
+    def expire_full_value(self):
+        key = self._full_value_cache_key.format(self.key)
+        cache.delete_pattern(key+'*')
+
+    @classmethod
+    def expire_nodes_full_value(cls, nodes=None):
+        key = cls._full_value_cache_key.format('*')
+        cache.delete_pattern(key+'*')
+
+
+class AssetsAmountMixin:
+    _assets_amount_cache_key = '_NODE_ASSETS_AMOUNT_{}'
+    _assets_amount = None
+    key = ''
+    cache_time = 3600 * 24 * 7
+
+    @property
+    def assets_amount(self):
+        """
+        获取节点下所有资产数量速度太慢，所以需要重写，使用cache等方案
+        :return:
+        """
+        if self._assets_amount is not None:
+            return self._assets_amount
+        cache_key = self._assets_amount_cache_key.format(self.key)
+        cached = cache.get(cache_key)
+        if cached is not None:
+            return cached
+        assets_amount = self.get_all_assets().count()
+        self.assets_amount = assets_amount
+        return assets_amount
+
+    @assets_amount.setter
+    def assets_amount(self, value):
+        self._assets_amount = value
+        cache_key = self._assets_amount_cache_key.format(self.key)
+        cache.set(cache_key, value, self.cache_time)
+
+    def expire_assets_amount(self):
+        ancestor_keys = self.get_ancestor_keys(with_self=True)
+        cache_keys = [self._assets_amount_cache_key.format(k) for k in
+                      ancestor_keys]
+        cache.delete_many(cache_keys)
+
+    @classmethod
+    def expire_nodes_assets_amount(cls, nodes=None):
+        key = cls._assets_amount_cache_key.format('*')
+        cache.delete_pattern(key)
+
+    @classmethod
+    def refresh_nodes(cls):
+        from ..utils import NodeUtil
+        util = NodeUtil(with_assets_amount=True)
+        util.set_assets_amount()
+        util.set_full_value()
+
+
+class Node(OrgModelMixin, FamilyMixin, FullValueMixin, AssetsAmountMixin):
     id = models.UUIDField(default=uuid.uuid4, primary_key=True)
     key = models.CharField(unique=True, max_length=64, verbose_name=_("Key"))  # '1:1:1:1'
-    value = models.CharField(max_length=128, unique=True, verbose_name=_("Value"))
+    value = models.CharField(max_length=128, verbose_name=_("Value"))
     child_mark = models.IntegerField(default=0)
     date_create = models.DateTimeField(auto_now_add=True)
 
+    objects = OrgManager.from_queryset(NodeQuerySet)()
+    is_node = True
+    _parents = None
+
+    class Meta:
+        verbose_name = _("Node")
+        ordering = ['key']
+
     def __str__(self):
-        return self.value
+        return self.full_value
+
+    def __eq__(self, other):
+        if not other:
+            return False
+        return self.id == other.id
+
+    def __gt__(self, other):
+        # if self.is_root() and not other.is_root():
+        #     return False
+        # elif not self.is_root() and other.is_root():
+        #     return True
+        self_key = [int(k) for k in self.key.split(':')]
+        other_key = [int(k) for k in other.key.split(':')]
+        self_parent_key = self_key[:-1]
+        other_parent_key = other_key[:-1]
+
+        if self_parent_key and other_parent_key and \
+                self_parent_key == other_parent_key:
+            return self.value > other.value
+        # if len(self_parent_key) < len(other_parent_key):
+        #     return True
+        # elif len(self_parent_key) > len(other_parent_key):
+        #     return False
+        return self_key > other_key
+
+    def __lt__(self, other):
+        return not self.__gt__(other)
 
     @property
     def name(self):
         return self.value
 
-    @property
-    def full_value(self):
-        if self == self.__class__.root():
-            return self.value
-        else:
-            return '{}/{}'.format(self.value, self.parent.full_value)
-
     @property
     def level(self):
         return len(self.key.split(':'))
@@ -40,80 +299,130 @@ class Node(models.Model):
         self.save()
         return "{}:{}".format(self.key, mark)
 
-    def create_child(self, value):
-        child_key = self.get_next_child_key()
-        child = self.__class__.objects.create(key=child_key, value=value)
-        return child
+    def get_next_child_preset_name(self):
+        name = ugettext("New node")
+        values = [
+            child.value[child.value.rfind(' '):]
+            for child in self.get_children()
+            if child.value.startswith(name)
+        ]
+        values = [int(value) for value in values if value.strip().isdigit()]
+        count = max(values) + 1 if values else 1
+        return '{} {}'.format(name, count)
 
-    def get_children(self):
-        return self.__class__.objects.filter(key__regex=r'{}:[0-9]+$'.format(self.key))
-
-    def get_all_children(self):
-        return self.__class__.objects.filter(key__startswith='{}:'.format(self.key))
-
-    def get_family(self):
-        children = list(self.get_all_children())
-        children.append(self)
-        return children
+    def create_child(self, value, _id=None):
+        with transaction.atomic():
+            child_key = self.get_next_child_key()
+            child = self.__class__.objects.create(id=_id, key=child_key, value=value)
+            return child
 
     def get_assets(self):
         from .asset import Asset
-        assets = Asset.objects.filter(nodes__id=self.id)
-        return assets
+        if self.is_default_node():
+            assets = Asset.objects.filter(Q(nodes__id=self.id) | Q(nodes__isnull=True))
+        else:
+            assets = Asset.objects.filter(nodes__id=self.id)
+        return assets.distinct()
 
-    def get_active_assets(self):
-        return self.get_assets().filter(is_active=True)
+    def get_valid_assets(self):
+        return self.get_assets().valid()
 
     def get_all_assets(self):
         from .asset import Asset
+        pattern = r'^{0}$|^{0}:'.format(self.key)
+        args = []
+        kwargs = {}
         if self.is_root():
-            assets = Asset.objects.all()
+            args.append(Q(nodes__key__regex=pattern) | Q(nodes=None))
         else:
-            nodes = self.get_family()
-            assets = Asset.objects.filter(nodes__in=nodes)
+            kwargs['nodes__key__regex'] = pattern
+        assets = Asset.objects.filter(*args, **kwargs).distinct()
         return assets
 
-    def get_all_active_assets(self):
-        return self.get_all_assets().filter(is_active=True)
+    def get_all_valid_assets(self):
+        return self.get_all_assets().valid()
+
+    def is_default_node(self):
+        return self.is_root() and self.key == '1'
 
     def is_root(self):
-        return self.key == '0'
-
-    @property
-    def parent(self):
-        if self.key == "0":
-            return self.__class__.root()
-        elif not self.key.startswith("0"):
-            return self.__class__.root()
-
-        parent_key = ":".join(self.key.split(":")[:-1])
-        try:
-            parent = self.__class__.objects.get(key=parent_key)
-        except Node.DoesNotExist:
-            return self.__class__.root()
+        if self.key.isdigit():
+            return True
         else:
-            return parent
+            return False
 
-    @parent.setter
-    def parent(self, parent):
-        self.key = parent.get_next_child_key()
-
-    @property
-    def ancestor(self):
-        if self.parent == self.__class__.root():
-            return [self.__class__.root()]
-        else:
-            return [self.parent, *tuple(self.parent.ancestor)]
-
-    @property
-    def ancestor_with_node(self):
-        ancestor = self.ancestor
-        ancestor.insert(0, self)
-        return ancestor
+    @classmethod
+    def create_root_node(cls):
+        # 如果使用current_org 在set_current_org时会死循环
+        _current_org = get_current_org()
+        with transaction.atomic():
+            if not _current_org.is_real():
+                return cls.default_node()
+            set_current_org(Organization.root())
+            org_nodes_roots = cls.objects.filter(key__regex=r'^[0-9]+$')
+            org_nodes_roots_keys = org_nodes_roots.values_list('key', flat=True) or ['1']
+            key = max([int(k) for k in org_nodes_roots_keys])
+            key = str(key + 1) if key != 0 else '2'
+            set_current_org(_current_org)
+            root = cls.objects.create(key=key, value=_current_org.name)
+            return root
 
     @classmethod
     def root(cls):
-        obj, created = cls.objects.get_or_create(
-            key='0', defaults={"key": '0', 'value': "ROOT"}
-        )
+        root = cls.objects.filter(key__regex=r'^[0-9]+$')
+        if root:
+            return root[0]
+        else:
+            return cls.create_root_node()
+
+    @classmethod
+    def default_node(cls):
+        defaults = {'value': 'Default'}
+        obj, created = cls.objects.get_or_create(defaults=defaults, key='1')
         return obj
+
+    def as_tree_node(self):
+        from common.tree import TreeNode
+        name = '{} ({})'.format(self.value, self.assets_amount)
+        data = {
+            'id': self.key,
+            'name': name,
+            'title': name,
+            'pId': self.parent_key,
+            'isParent': True,
+            'open': self.is_root(),
+            'meta': {
+                'node': {
+                    "id": self.id,
+                    "name": self.name,
+                    "value": self.value,
+                    "key": self.key,
+                    "assets_amount": self.assets_amount,
+                },
+                'type': 'node'
+            }
+        }
+        tree_node = TreeNode(**data)
+        return tree_node
+
+    def delete(self, using=None, keep_parents=False):
+        if self.children or self.get_assets():
+            return
+        return super().delete(using=using, keep_parents=keep_parents)
+
+    @classmethod
+    def get_queryset(cls):
+        from ..utils import NodeUtil
+        util = NodeUtil()
+        return sorted(util.nodes)
+
+    @classmethod
+    def generate_fake(cls, count=100):
+        import random
+        org = get_current_org()
+        if not org or not org.is_real():
+            Organization.default().change_to()
+
+        for i in range(count):
+            node = random.choice(cls.objects.all())
+            node.create_child('Node {}'.format(i))

apps/assets/models/user.py

@@ -2,138 +2,21 @@
 # -*- coding: utf-8 -*-
 #
 
-import os
 import logging
-import uuid
-from hashlib import md5
 
-import sshpubkeys
-from django.core.cache import cache
 from django.db import models
 from django.utils.translation import ugettext_lazy as _
-from django.conf import settings
+from django.core.validators import MinValueValidator, MaxValueValidator
 
-from common.utils import get_signer, ssh_key_string_to_obj, ssh_key_gen
-from .utils import private_key_validator
-from ..const import SYSTEM_USER_CONN_CACHE_KEY
+from common.utils import get_signer
+from .base import AssetUser
 
 
-__all__ = ['AdminUser', 'SystemUser',]
+__all__ = ['AdminUser', 'SystemUser']
 logger = logging.getLogger(__name__)
 signer = get_signer()
 
 
-class AssetUser(models.Model):
-    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
-    name = models.CharField(max_length=128, unique=True, verbose_name=_('Name'))
-    username = models.CharField(max_length=128, verbose_name=_('Username'))
-    _password = models.CharField(max_length=256, blank=True, null=True, verbose_name=_('Password'))
-    _private_key = models.TextField(max_length=4096, blank=True, null=True, verbose_name=_('SSH private key'), validators=[private_key_validator, ])
-    _public_key = models.TextField(max_length=4096, blank=True, verbose_name=_('SSH public key'))
-    comment = models.TextField(blank=True, verbose_name=_('Comment'))
-    date_created = models.DateTimeField(auto_now_add=True)
-    date_updated = models.DateTimeField(auto_now=True)
-    created_by = models.CharField(max_length=128, null=True, verbose_name=_('Created by'))
-
-    @property
-    def password(self):
-        if self._password:
-            return signer.unsign(self._password)
-        else:
-            return None
-
-    @password.setter
-    def password(self, password_raw):
-        raise AttributeError("Using set_auth do that")
-        # self._password = signer.sign(password_raw)
-
-    @property
-    def private_key(self):
-        if self._private_key:
-            return signer.unsign(self._private_key)
-
-    @private_key.setter
-    def private_key(self, private_key_raw):
-        raise AttributeError("Using set_auth do that")
-        # self._private_key = signer.sign(private_key_raw)
-
-    @property
-    def private_key_obj(self):
-        if self._private_key:
-            key_str = signer.unsign(self._private_key)
-            return ssh_key_string_to_obj(key_str, password=self.password)
-        else:
-            return None
-
-    @property
-    def private_key_file(self):
-        if not self.private_key_obj:
-            return None
-        project_dir = settings.PROJECT_DIR
-        tmp_dir = os.path.join(project_dir, 'tmp')
-        key_str = signer.unsign(self._private_key)
-        key_name = '.' + md5(key_str.encode('utf-8')).hexdigest()
-        key_path = os.path.join(tmp_dir, key_name)
-        if not os.path.exists(key_path):
-            self.private_key_obj.write_private_key_file(key_path)
-            os.chmod(key_path, 0o400)
-        return key_path
-
-    @property
-    def public_key(self):
-        key = signer.unsign(self._public_key)
-        if key:
-            return key
-        else:
-            return None
-
-    @property
-    def public_key_obj(self):
-        if self.public_key:
-            try:
-                return sshpubkeys.SSHKey(self.public_key)
-            except TabError:
-                pass
-        return None
-
-    def set_auth(self, password=None, private_key=None, public_key=None):
-        update_fields = []
-        if password:
-            self._password = signer.sign(password)
-            update_fields.append('_password')
-        if private_key:
-            self._private_key = signer.sign(private_key)
-            update_fields.append('_private_key')
-        if public_key:
-            self._public_key = signer.sign(public_key)
-            update_fields.append('_public_key')
-
-        if update_fields:
-            self.save(update_fields=update_fields)
-
-    def auto_gen_auth(self):
-        password = str(uuid.uuid4())
-        private_key, public_key = ssh_key_gen(
-            username=self.name, password=password
-        )
-        self.set_auth(password=password,
-                      private_key=private_key,
-                      public_key=public_key)
-
-    def _to_secret_json(self):
-        """Push system user use it"""
-        return {
-            'name': self.name,
-            'username': self.username,
-            'password': self.password,
-            'public_key': self.public_key,
-            'private_key': self.private_key_file,
-        }
-
-    class Meta:
-        abstract = True
-
-
 class AdminUser(AssetUser):
     """
     A privileged user that ansible can use it to push system user and so on
@@ -146,6 +29,8 @@ class AdminUser(AssetUser):
     become_method = models.CharField(choices=BECOME_METHOD_CHOICES, default='sudo', max_length=4)
     become_user = models.CharField(default='root', max_length=64)
     _become_pass = models.CharField(default='', max_length=128)
+    CONNECTIVITY_CACHE_KEY = '_ADMIN_USER_CONNECTIVE_{}'
+    _prefer = "admin_user"
 
     def __str__(self):
         return self.name
@@ -174,16 +59,9 @@ class AdminUser(AssetUser):
             info = None
         return info
 
-    def get_related_assets(self):
-        assets = self.asset_set.all()
-        return assets
-
-    @property
-    def assets_amount(self):
-        return self.get_related_assets().count()
-
     class Meta:
         ordering = ['name']
+        unique_together = [('name', 'org_id')]
         verbose_name = _("Admin user")
 
     @classmethod
@@ -208,22 +86,36 @@ class AdminUser(AssetUser):
 
 
 class SystemUser(AssetUser):
-    SSH_PROTOCOL = 'ssh'
-    RDP_PROTOCOL = 'rdp'
+    PROTOCOL_SSH = 'ssh'
+    PROTOCOL_RDP = 'rdp'
+    PROTOCOL_TELNET = 'telnet'
+    PROTOCOL_VNC = 'vnc'
     PROTOCOL_CHOICES = (
-        (SSH_PROTOCOL, 'ssh'),
-        (RDP_PROTOCOL, 'rdp'),
+        (PROTOCOL_SSH, 'ssh'),
+        (PROTOCOL_RDP, 'rdp'),
+        (PROTOCOL_TELNET, 'telnet (beta)'),
+        (PROTOCOL_VNC, 'vnc'),
+    )
+
+    LOGIN_AUTO = 'auto'
+    LOGIN_MANUAL = 'manual'
+    LOGIN_MODE_CHOICES = (
+        (LOGIN_AUTO, _('Automatic login')),
+        (LOGIN_MANUAL, _('Manually login'))
     )
 
     nodes = models.ManyToManyField('assets.Node', blank=True, verbose_name=_("Nodes"))
-    priority = models.IntegerField(default=10, verbose_name=_("Priority"))
+    assets = models.ManyToManyField('assets.Asset', blank=True, verbose_name=_("Assets"))
+    priority = models.IntegerField(default=20, verbose_name=_("Priority"), validators=[MinValueValidator(1), MaxValueValidator(100)])
     protocol = models.CharField(max_length=16, choices=PROTOCOL_CHOICES, default='ssh', verbose_name=_('Protocol'))
     auto_push = models.BooleanField(default=True, verbose_name=_('Auto push'))
-    sudo = models.TextField(default='/sbin/ifconfig', verbose_name=_('Sudo'))
+    sudo = models.TextField(default='/bin/whoami', verbose_name=_('Sudo'))
     shell = models.CharField(max_length=64,  default='/bin/bash', verbose_name=_('Shell'))
+    login_mode = models.CharField(choices=LOGIN_MODE_CHOICES, default=LOGIN_AUTO, max_length=10, verbose_name=_('Login mode'))
+    cmd_filters = models.ManyToManyField('CommandFilter', related_name='system_users', verbose_name=_("Command filter"), blank=True)
 
     def __str__(self):
-        return self.name
+        return '{0.name}({0.username})'.format(self)
 
     def to_json(self):
         return {
@@ -236,33 +128,35 @@ class SystemUser(AssetUser):
         }
 
     @property
-    def assets(self):
-        assets = set()
-        for node in self.nodes.all():
-            assets.update(set(node.get_all_assets()))
-        return assets
-
-    @property
-    def assets_connective(self):
-        _result = cache.get(SYSTEM_USER_CONN_CACHE_KEY.format(self.name), {})
-        return _result
-
-    @property
-    def unreachable_assets(self):
-        return list(self.assets_connective.get('dark', {}).keys())
-
-    @property
-    def reachable_assets(self):
-        return self.assets_connective.get('contacted', [])
+    def login_mode_display(self):
+        return self.get_login_mode_display()
 
     def is_need_push(self):
-        if self.auto_push and self.protocol == self.__class__.SSH_PROTOCOL:
+        if self.auto_push and self.protocol in [self.PROTOCOL_SSH, self.PROTOCOL_RDP]:
             return True
         else:
             return False
 
+    @property
+    def cmd_filter_rules(self):
+        from .cmd_filter import CommandFilterRule
+        rules = CommandFilterRule.objects.filter(
+            filter__in=self.cmd_filters.all()
+        ).distinct()
+        return rules
+
+    def is_command_can_run(self, command):
+        for rule in self.cmd_filter_rules:
+            action, matched_cmd = rule.match(command)
+            if action == rule.ACTION_ALLOW:
+                return True, None
+            elif action == rule.ACTION_DENY:
+                return False, matched_cmd
+        return True, None
+
     class Meta:
         ordering = ['name']
+        unique_together = [('name', 'org_id')]
         verbose_name = _("System user")
 
     @classmethod
@@ -284,6 +178,3 @@ class SystemUser(AssetUser):
             except IntegrityError:
                 print('Error continue')
                 continue
-
-
-

apps/assets/models/utils.py

@@ -2,23 +2,29 @@
 # -*- coding: utf-8 -*-
 #
 
+from django.utils import timezone
+from django.core.cache import cache
 from django.core.exceptions import ValidationError
+from django.utils.translation import ugettext_lazy as _
+
 from common.utils import validate_ssh_private_key
 
 
-__all__ = ['init_model', 'generate_fake']
+__all__ = [
+    'init_model', 'generate_fake', 'private_key_validator', 'Connectivity',
+]
 
 
 def init_model():
-    from . import Cluster, SystemUser, AdminUser, AssetGroup, Asset
-    for cls in [Cluster, SystemUser, AdminUser, AssetGroup, Asset]:
+    from . import SystemUser, AdminUser, Asset
+    for cls in [SystemUser, AdminUser, Asset]:
         if hasattr(cls, 'initial'):
             cls.initial()
 
 
 def generate_fake():
-    from . import Cluster, SystemUser, AdminUser, AssetGroup, Asset
-    for cls in [Cluster, SystemUser, AdminUser, AssetGroup, Asset]:
+    from . import SystemUser, AdminUser,  Asset
+    for cls in [SystemUser, AdminUser, Asset]:
         if hasattr(cls, 'generate_fake'):
             cls.generate_fake()
 
@@ -31,5 +37,72 @@ def private_key_validator(value):
         )
 
 
-if __name__ == '__main__':
-    pass
+class Connectivity:
+    UNREACHABLE, REACHABLE, UNKNOWN = range(0, 3)
+    CONNECTIVITY_CHOICES = (
+        (UNREACHABLE, _("Unreachable")),
+        (REACHABLE, _('Reachable')),
+        (UNKNOWN, _("Unknown")),
+    )
+
+    status = UNKNOWN
+    datetime = timezone.now()
+
+    def __init__(self, status, datetime):
+        self.status = status
+        self.datetime = datetime
+
+    def display(self):
+        return dict(self.__class__.CONNECTIVITY_CHOICES).get(self.status)
+
+    def is_reachable(self):
+        return self.status == self.REACHABLE
+
+    def is_unreachable(self):
+        return self.status == self.UNREACHABLE
+
+    def is_unknown(self):
+        return self.status == self.UNKNOWN
+
+    @classmethod
+    def unreachable(cls):
+        return cls(cls.UNREACHABLE, timezone.now())
+
+    @classmethod
+    def reachable(cls):
+        return cls(cls.REACHABLE, timezone.now())
+
+    @classmethod
+    def unknown(cls):
+        return cls(cls.UNKNOWN, timezone.now())
+
+    @classmethod
+    def set(cls, key, value, ttl=0):
+        cache.set(key, value, ttl)
+
+    @classmethod
+    def get(cls, key):
+        value = cache.get(key, cls.unknown())
+        if not isinstance(value, cls):
+            value = cls.unknown()
+        return value
+
+    @classmethod
+    def set_unreachable(cls, key, ttl=0):
+        cls.set(key, cls.unreachable(), ttl)
+
+    @classmethod
+    def set_reachable(cls, key, ttl=0):
+        cls.set(key, cls.reachable(), ttl)
+
+    def __eq__(self, other):
+        return self.status == other.status
+
+    def __gt__(self, other):
+        return self.status > other.status
+
+    def __lt__(self, other):
+        return not self.__gt__(other)
+
+    def __str__(self):
+        return self.display()

apps/assets/serializers/__init__.py

@@ -6,3 +6,6 @@ from .admin_user import *
 from .label import *
 from .system_user import *
 from .node import *
+from .domain import *
+from .cmd_filter import *
+from .asset_user import *

apps/assets/serializers/admin_user.py

@@ -1,42 +1,47 @@
 # -*- coding: utf-8 -*-
 #
-from django.core.cache import cache
+from django.utils.translation import ugettext_lazy as _
 from rest_framework import serializers
+
+from common.serializers import AdaptedBulkListSerializer
+
 from ..models import Node, AdminUser
-from ..const import ADMIN_USER_CONN_CACHE_KEY
+from orgs.mixins import BulkOrgResourceModelSerializer
+
+from .base import AuthSerializer
 
 
-class AdminUserSerializer(serializers.ModelSerializer):
+class AdminUserSerializer(BulkOrgResourceModelSerializer):
     """
     管理用户
     """
-    assets_amount = serializers.SerializerMethodField()
-    unreachable_amount = serializers.SerializerMethodField()
-    reachable_amount = serializers.SerializerMethodField()
+
+    class Meta:
+        list_serializer_class = AdaptedBulkListSerializer
+        model = AdminUser
+        fields = [
+            'id', 'name', 'username', 'password', 'private_key', 'public_key',
+            'comment', 'connectivity_amount', 'assets_amount',
+            'date_created', 'date_updated', 'created_by',
+        ]
+
+        extra_kwargs = {
+            'password': {"write_only": True},
+            'private_key': {"write_only": True},
+            'public_key': {"write_only": True},
+            'date_created': {'read_only': True},
+            'date_updated': {'read_only': True},
+            'created_by': {'read_only': True},
+            'assets_amount': {'label': _('Asset')},
+            'connectivity_amount': {'label': _('Connectivity')},
+        }
+
+
+class AdminUserAuthSerializer(AuthSerializer):
 
     class Meta:
         model = AdminUser
-        fields = '__all__'
-
-    @staticmethod
-    def get_unreachable_amount(obj):
-        data = cache.get(ADMIN_USER_CONN_CACHE_KEY.format(obj.name))
-        if data:
-            return len(data.get('dark'))
-        else:
-            return 0
-
-    @staticmethod
-    def get_reachable_amount(obj):
-        data = cache.get(ADMIN_USER_CONN_CACHE_KEY.format(obj.name))
-        if data:
-            return len(data.get('contacted'))
-        else:
-            return 0
-
-    @staticmethod
-    def get_assets_amount(obj):
-        return obj.assets_amount
+        fields = ['password', 'private_key']
 
 
 class ReplaceNodeAdminUserSerializer(serializers.ModelSerializer):
@@ -50,3 +55,7 @@ class ReplaceNodeAdminUserSerializer(serializers.ModelSerializer):
     class Meta:
         model = AdminUser
         fields = ['id', 'nodes']
+
+
+class TaskIDSerializer(serializers.Serializer):
+    task = serializers.CharField(read_only=True)

apps/assets/serializers/asset.py

@@ -1,62 +1,132 @@
 # -*- coding: utf-8 -*-
 #
 from rest_framework import serializers
-from rest_framework_bulk.serializers import BulkListSerializer
+from django.db.models import Prefetch
+from django.utils.translation import ugettext_lazy as _
 
-from common.mixins import BulkSerializerMixin
-from ..models import Asset
-from .system_user import AssetSystemUserSerializer
+from orgs.mixins import BulkOrgResourceModelSerializer
+from common.serializers import AdaptedBulkListSerializer
+from ..models import Asset, Node, Label
+from .base import ConnectivitySerializer
+
+__all__ = [
+    'AssetSerializer', 'AssetSimpleSerializer',
+    'ProtocolsField',
+]
 
 
-class AssetSerializer(BulkSerializerMixin, serializers.ModelSerializer):
+class ProtocolField(serializers.RegexField):
+    protocols = '|'.join(dict(Asset.PROTOCOL_CHOICES).keys())
+    default_error_messages = {
+        'invalid': _('Protocol format should {}/{}'.format(protocols, '1-65535'))
+    }
+    regex = r'^(%s)/(\d{1,5})$' % protocols
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(self.regex, **kwargs)
+
+
+def validate_duplicate_protocols(values):
+    errors = []
+    names = []
+
+    for value in values:
+        if not value or '/' not in value:
+            continue
+        name = value.split('/')[0]
+        if name in names:
+            errors.append(_("Protocol duplicate: {}").format(name))
+        names.append(name)
+        errors.append('')
+    if any(errors):
+        raise serializers.ValidationError(errors)
+
+
+class ProtocolsField(serializers.ListField):
+    default_validators = [validate_duplicate_protocols]
+
+    def __init__(self, *args, **kwargs):
+        kwargs['child'] = ProtocolField()
+        kwargs['allow_null'] = True
+        kwargs['allow_empty'] = True
+        kwargs['min_length'] = 1
+        kwargs['max_length'] = 4
+        super().__init__(*args, **kwargs)
+
+    def to_representation(self, value):
+        if not value:
+            return []
+        return value.split(' ')
+
+
+class AssetSerializer(BulkOrgResourceModelSerializer):
+    protocols = ProtocolsField(label=_('Protocols'), required=False)
+    connectivity = ConnectivitySerializer(read_only=True, label=_("Connectivity"))
+
     """
     资产的数据结构
     """
-
     class Meta:
         model = Asset
-        list_serializer_class = BulkListSerializer
-        fields = '__all__'
-        validators = []  # If not set to [], partial bulk update will be error
-
-    def get_field_names(self, declared_fields, info):
-        fields = super().get_field_names(declared_fields, info)
-        fields.extend([
-            'hardware_info', 'is_connective',
-        ])
-        return fields
-
-
-class AssetGrantedSerializer(serializers.ModelSerializer):
-    """
-    被授权资产的数据结构
-    """
-    system_users_granted = AssetSystemUserSerializer(many=True, read_only=True)
-    system_users_join = serializers.SerializerMethodField()
-
-    class Meta:
-        model = Asset
-        fields = (
-            "id", "hostname", "ip", "port", "system_users_granted",
-            "is_active", "system_users_join", "os",
-            "platform", "comment"
+        list_serializer_class = AdaptedBulkListSerializer
+        fields = [
+            'id', 'ip', 'hostname', 'protocol', 'port',
+            'protocols', 'platform', 'is_active', 'public_ip', 'domain',
+            'admin_user', 'nodes', 'labels', 'number', 'vendor', 'model', 'sn',
+            'cpu_model', 'cpu_count', 'cpu_cores', 'cpu_vcpus', 'memory',
+            'disk_total', 'disk_info', 'os', 'os_version', 'os_arch',
+            'hostname_raw', 'comment', 'created_by', 'date_created',
+            'hardware_info', 'connectivity',
+        ]
+        read_only_fields = (
+            'vendor', 'model', 'sn', 'cpu_model', 'cpu_count',
+            'cpu_cores', 'cpu_vcpus', 'memory', 'disk_total', 'disk_info',
+            'os', 'os_version', 'os_arch', 'hostname_raw',
+            'created_by', 'date_created',
         )
+        extra_kwargs = {
+            'protocol': {'write_only': True},
+            'port': {'write_only': True},
+            'hardware_info': {'label': _('Hardware info')},
+            'org_name': {'label': _('Org name')}
+        }
 
-    @staticmethod
-    def get_system_users_join(obj):
-        system_users = [s.username for s in obj.system_users_granted]
-        return ', '.join(system_users)
+    @classmethod
+    def setup_eager_loading(cls, queryset):
+        """ Perform necessary eager loading of data. """
+        queryset = queryset.prefetch_related(
+            Prefetch('nodes', queryset=Node.objects.all().only('id')),
+            Prefetch('labels', queryset=Label.objects.all().only('id')),
+        ).select_related('admin_user', 'domain')
+        return queryset
+
+    def compatible_with_old_protocol(self, validated_data):
+        protocols_data = validated_data.pop("protocols", [])
+
+        # 兼容老的api
+        name = validated_data.get("protocol")
+        port = validated_data.get("port")
+        if not protocols_data and name and port:
+            protocols_data.insert(0, '/'.join([name, str(port)]))
+        elif not name and not port and protocols_data:
+            protocol = protocols_data[0].split('/')
+            validated_data["protocol"] = protocol[0]
+            validated_data["port"] = int(protocol[1])
+        if validated_data:
+            validated_data["protocols"] = ' '.join(protocols_data)
+
+    def create(self, validated_data):
+        self.compatible_with_old_protocol(validated_data)
+        instance = super().create(validated_data)
+        return instance
+
+    def update(self, instance, validated_data):
+        self.compatible_with_old_protocol(validated_data)
+        return super().update(instance, validated_data)
 
 
-class MyAssetGrantedSerializer(AssetGrantedSerializer):
-    """
-    普通用户获取授权的资产定义的数据结构
-    """
+class AssetSimpleSerializer(serializers.ModelSerializer):
 
     class Meta:
         model = Asset
-        fields = (
-            "id", "hostname", "system_users_granted",
-            "is_active", "system_users_join",
-            "os", "platform", "comment",
-        )
+        fields = ['id', 'hostname', 'ip', 'connectivity', 'port']

apps/assets/serializers/asset_user.py

@@ -0,0 +1,96 @@
+# -*- coding: utf-8 -*-
+#
+
+from django.utils.translation import ugettext as _
+from rest_framework import serializers
+
+from common.utils import validate_ssh_private_key
+from common.serializers import AdaptedBulkListSerializer
+from orgs.mixins import BulkOrgResourceModelSerializer
+from ..models import AuthBook, Asset
+from ..backends import AssetUserManager
+from .base import ConnectivitySerializer
+
+
+__all__ = [
+    'AssetUserSerializer', 'AssetUserAuthInfoSerializer',
+    'AssetUserExportSerializer', 'AssetUserPushSerializer',
+]
+
+
+class BasicAssetSerializer(serializers.ModelSerializer):
+    class Meta:
+        model = Asset
+        fields = ['hostname', 'ip']
+
+
+class AssetUserSerializer(BulkOrgResourceModelSerializer):
+    hostname = serializers.CharField(read_only=True, label=_("Hostname"))
+    ip = serializers.CharField(read_only=True, label=_("IP"))
+    connectivity = ConnectivitySerializer(read_only=True, label=_("Connectivity"))
+
+    backend = serializers.CharField(read_only=True, label=_("Backend"))
+
+    class Meta:
+        model = AuthBook
+        list_serializer_class = AdaptedBulkListSerializer
+        read_only_fields = (
+            'date_created', 'date_updated', 'created_by',
+            'is_latest', 'version', 'connectivity',
+        )
+        fields = [
+            "id", "hostname", "ip", "username", "password", "asset", "version",
+            "is_latest", "connectivity", "backend",
+            "date_created", "date_updated", "private_key", "public_key",
+        ]
+        extra_kwargs = {
+            'username': {'required': True},
+            'password': {'write_only': True},
+            'private_key': {'write_only': True},
+            'public_key': {'write_only': True},
+        }
+
+    def validate_private_key(self, key):
+        password = self.initial_data.get("password")
+        valid = validate_ssh_private_key(key, password)
+        if not valid:
+            raise serializers.ValidationError(_("private key invalid"))
+        return key
+
+    def create(self, validated_data):
+        if not validated_data.get("name") and validated_data.get("username"):
+            validated_data["name"] = validated_data["username"]
+        instance = AssetUserManager.create(**validated_data)
+        return instance
+
+
+class AssetUserExportSerializer(AssetUserSerializer):
+    password = serializers.CharField(
+        max_length=256, allow_blank=True, allow_null=True,
+        required=False, label=_('Password')
+    )
+    public_key = serializers.CharField(
+        max_length=4096, allow_blank=True, allow_null=True,
+        required=False, label=_('Public key')
+    )
+    private_key = serializers.CharField(
+        max_length=4096, allow_blank=True, allow_null=True,
+        required=False, label=_('Private key')
+    )
+
+
+class AssetUserAuthInfoSerializer(serializers.ModelSerializer):
+    class Meta:
+        model = AuthBook
+        fields = ['password', 'private_key', 'public_key']
+
+
+class AssetUserPushSerializer(serializers.Serializer):
+    asset = serializers.PrimaryKeyRelatedField(queryset=Asset.objects.all(), label=_("Asset"))
+    username = serializers.CharField(max_length=1024)
+
+    def create(self, validated_data):
+        pass
+
+    def update(self, instance, validated_data):
+        pass

apps/assets/serializers/base.py

@@ -0,0 +1,31 @@
+# -*- coding: utf-8 -*-
+#
+
+from rest_framework import serializers
+from common.utils import ssh_pubkey_gen
+
+
+class AuthSerializer(serializers.ModelSerializer):
+    password = serializers.CharField(required=False, allow_blank=True, allow_null=True, max_length=1024)
+    private_key = serializers.CharField(required=False, allow_blank=True, allow_null=True, max_length=4096)
+
+    def gen_keys(self, private_key=None, password=None):
+        if private_key is None:
+            return None, None
+        public_key = ssh_pubkey_gen(private_key=private_key, password=password)
+        return private_key, public_key
+
+    def save(self, **kwargs):
+        password = self.validated_data.pop('password', None) or None
+        private_key = self.validated_data.pop('private_key', None) or None
+        self.instance = super().save(**kwargs)
+        if password or private_key:
+            private_key, public_key = self.gen_keys(private_key, password)
+            self.instance.set_auth(password=password, private_key=private_key,
+                                   public_key=public_key)
+        return self.instance
+
+
+class ConnectivitySerializer(serializers.Serializer):
+    status = serializers.IntegerField()
+    datetime = serializers.DateTimeField()

apps/assets/serializers/cluster.py

@@ -1,46 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-
-from rest_framework import serializers
-from common.mixins import BulkSerializerMixin
-from ..models import Asset, Cluster
-
-
-class ClusterUpdateAssetsSerializer(serializers.ModelSerializer):
-    """
-    集群更新资产数据结构
-    """
-    assets = serializers.PrimaryKeyRelatedField(many=True, queryset=Asset.objects.all())
-
-    class Meta:
-        model = Cluster
-        fields = ['id', 'assets']
-
-
-class ClusterSerializer(BulkSerializerMixin, serializers.ModelSerializer):
-    """
-    cluster
-    """
-    assets_amount = serializers.SerializerMethodField()
-    admin_user_name = serializers.SerializerMethodField()
-    assets = serializers.PrimaryKeyRelatedField(many=True, queryset=Asset.objects.all())
-    system_users = serializers.SerializerMethodField()
-
-    class Meta:
-        model = Cluster
-        fields = '__all__'
-
-    @staticmethod
-    def get_assets_amount(obj):
-        return obj.assets.count()
-
-    @staticmethod
-    def get_admin_user_name(obj):
-        try:
-            return obj.admin_user.name
-        except AttributeError:
-            return ''
-
-    @staticmethod
-    def get_system_users(obj):
-        return ', '.join(obj.name for obj in obj.systemuser_set.all())

apps/assets/serializers/cmd_filter.py

@@ -0,0 +1,36 @@
+# -*- coding: utf-8 -*-
+#
+import re
+from rest_framework import serializers
+
+from common.fields import ChoiceDisplayField
+from common.serializers import AdaptedBulkListSerializer
+from ..models import CommandFilter, CommandFilterRule, SystemUser
+from orgs.mixins import BulkOrgResourceModelSerializer
+
+
+class CommandFilterSerializer(BulkOrgResourceModelSerializer):
+    rules = serializers.PrimaryKeyRelatedField(queryset=CommandFilterRule.objects.all(), many=True)
+    system_users = serializers.PrimaryKeyRelatedField(queryset=SystemUser.objects.all(), many=True)
+
+    class Meta:
+        model = CommandFilter
+        list_serializer_class = AdaptedBulkListSerializer
+        fields = '__all__'
+
+
+class CommandFilterRuleSerializer(BulkOrgResourceModelSerializer):
+    serializer_choice_field = ChoiceDisplayField
+    invalid_pattern = re.compile(r'[\.\*\+\[\\\?\{\}\^\$\|\(\)\#\<\>]')
+
+    class Meta:
+        model = CommandFilterRule
+        fields = '__all__'
+        list_serializer_class = AdaptedBulkListSerializer
+
+    def validate_content(self, content):
+        if self.invalid_pattern.search(content):
+            invalid_char = self.invalid_pattern.pattern.replace('\\', '')
+            msg = _("Content should not be contain: {}").format(invalid_char)
+            raise serializers.ValidationError(msg)
+        return content

apps/assets/serializers/domain.py

@@ -0,0 +1,54 @@
+# -*- coding: utf-8 -*-
+#
+from rest_framework import serializers
+
+from common.serializers import AdaptedBulkListSerializer
+from orgs.mixins import BulkOrgResourceModelSerializer
+
+from ..models import Domain, Gateway
+
+
+class DomainSerializer(BulkOrgResourceModelSerializer):
+    asset_count = serializers.SerializerMethodField()
+    gateway_count = serializers.SerializerMethodField()
+
+    class Meta:
+        model = Domain
+        fields = '__all__'
+        list_serializer_class = AdaptedBulkListSerializer
+
+    @staticmethod
+    def get_asset_count(obj):
+        return obj.assets.count()
+
+    @staticmethod
+    def get_gateway_count(obj):
+        return obj.gateway_set.all().count()
+
+
+class GatewaySerializer(BulkOrgResourceModelSerializer):
+    class Meta:
+        model = Gateway
+        list_serializer_class = AdaptedBulkListSerializer
+        fields = [
+            'id', 'name', 'ip', 'port', 'protocol', 'username',
+            'domain', 'is_active', 'date_created', 'date_updated',
+            'created_by', 'comment',
+        ]
+
+
+class GatewayWithAuthSerializer(GatewaySerializer):
+    def get_field_names(self, declared_fields, info):
+        fields = super().get_field_names(declared_fields, info)
+        fields.extend(
+            ['password', 'private_key']
+        )
+        return fields
+
+
+class DomainWithGatewaySerializer(BulkOrgResourceModelSerializer):
+    gateways = GatewayWithAuthSerializer(many=True, read_only=True)
+
+    class Meta:
+        model = Domain
+        fields = '__all__'

apps/assets/serializers/label.py

@@ -1,18 +1,20 @@
 # -*- coding: utf-8 -*-
 #
 from rest_framework import serializers
-from rest_framework_bulk.serializers import BulkListSerializer
+
+from common.serializers import AdaptedBulkListSerializer
+from orgs.mixins import BulkOrgResourceModelSerializer
 
 from ..models import Label
 
 
-class LabelSerializer(serializers.ModelSerializer):
+class LabelSerializer(BulkOrgResourceModelSerializer):
     asset_count = serializers.SerializerMethodField()
 
     class Meta:
         model = Label
         fields = '__all__'
-        list_serializer_class = BulkListSerializer
+        list_serializer_class = AdaptedBulkListSerializer
 
     @staticmethod
     def get_asset_count(obj):
@@ -24,7 +26,7 @@ class LabelSerializer(serializers.ModelSerializer):
         return fields
 
 
-class LabelDistinctSerializer(serializers.ModelSerializer):
+class LabelDistinctSerializer(BulkOrgResourceModelSerializer):
     value = serializers.SerializerMethodField()
 
     class Meta:

apps/assets/serializers/node.py

@@ -1,63 +1,38 @@
 # -*- coding: utf-8 -*-
 from rest_framework import serializers
-from rest_framework_bulk.serializers import BulkListSerializer
+from django.utils.translation import ugettext as _
 
-from common.mixins import BulkSerializerMixin
+from orgs.mixins import BulkOrgResourceModelSerializer
 from ..models import Asset, Node
-from .asset import AssetGrantedSerializer
 
 
-class NodeGrantedSerializer(BulkSerializerMixin, serializers.ModelSerializer):
-    """
-    授权资产组
-    """
-    assets_granted = AssetGrantedSerializer(many=True, read_only=True)
-    assets_amount = serializers.SerializerMethodField()
-    parent = serializers.SerializerMethodField()
-    name = serializers.SerializerMethodField()
+__all__ = [
+    'NodeSerializer', "NodeAddChildrenSerializer",
+    "NodeAssetsSerializer",
+]
+
+
+class NodeSerializer(BulkOrgResourceModelSerializer):
+    assets_amount = serializers.IntegerField(read_only=True)
 
     class Meta:
         model = Node
         fields = [
-            'id', 'key', 'name', 'value', 'parent',
-            'assets_granted', 'assets_amount',
+            'id', 'key', 'value', 'assets_amount', 'org_id',
+        ]
+        read_only_fields = [
+            'key', 'assets_amount', 'org_id',
         ]
 
-    @staticmethod
-    def get_assets_amount(obj):
-        return len(obj.assets_granted)
-
-    @staticmethod
-    def get_name(obj):
-        return obj.name
-
-    @staticmethod
-    def get_parent(obj):
-        return obj.parent.id
-
-
-class NodeSerializer(serializers.ModelSerializer):
-    parent = serializers.SerializerMethodField()
-    assets_amount = serializers.SerializerMethodField()
-
-    class Meta:
-        model = Node
-        fields = ['id', 'key', 'value', 'parent', 'assets_amount']
-        list_serializer_class = BulkListSerializer
-
-    @staticmethod
-    def get_parent(obj):
-        return obj.parent.id
-
-    @staticmethod
-    def get_assets_amount(obj):
-        return obj.get_all_assets().count()
-
-    def get_fields(self):
-        fields = super().get_fields()
-        field = fields["key"]
-        field.required = False
-        return fields
+    def validate_value(self, data):
+        instance = self.instance if self.instance else Node.root()
+        children = instance.parent.get_children()
+        children_values = [node.value for node in children if node != instance]
+        if data in children_values:
+            raise serializers.ValidationError(
+                _('The same level node name cannot be the same')
+            )
+        return data
 
 
 class NodeAssetsSerializer(serializers.ModelSerializer):
@@ -70,3 +45,4 @@ class NodeAssetsSerializer(serializers.ModelSerializer):
 
 class NodeAddChildrenSerializer(serializers.Serializer):
     nodes = serializers.ListField()
+

apps/assets/serializers/system_user.py

@@ -1,64 +1,57 @@
 from rest_framework import serializers
 
+from django.utils.translation import ugettext_lazy as _
+
+from common.serializers import AdaptedBulkListSerializer
+from orgs.mixins import BulkOrgResourceModelSerializer
 from ..models import SystemUser
+from .base import AuthSerializer
 
 
-class SystemUserSerializer(serializers.ModelSerializer):
+class SystemUserSerializer(BulkOrgResourceModelSerializer):
     """
     系统用户
     """
-    unreachable_amount = serializers.SerializerMethodField()
-    reachable_amount = serializers.SerializerMethodField()
-    unreachable_assets = serializers.SerializerMethodField()
-    reachable_assets = serializers.SerializerMethodField()
-    assets_amount = serializers.SerializerMethodField()
 
     class Meta:
         model = SystemUser
-        exclude = ('_password', '_private_key', '_public_key')
+        list_serializer_class = AdaptedBulkListSerializer
+        fields = [
+            'id', 'name', 'username', 'password', 'public_key', 'private_key',
+            'login_mode', 'login_mode_display', 'priority', 'protocol',
+            'auto_push', 'cmd_filters', 'sudo', 'shell', 'comment', 'nodes',
+            'assets_amount', 'connectivity_amount'
+        ]
+        extra_kwargs = {
+            'password': {"write_only": True},
+            'public_key': {"write_only": True},
+            'private_key': {"write_only": True},
+            'assets_amount': {'label': _('Asset')},
+            'connectivity_amount': {'label': _('Connectivity')},
+            'login_mode_display': {'label': _('Login mode display')},
+            'created_by': {'read_only': True},
+        }
 
-    @staticmethod
-    def get_unreachable_assets(obj):
-        return obj.unreachable_assets
-
-    @staticmethod
-    def get_reachable_assets(obj):
-        return obj.reachable_assets
-
-    def get_unreachable_amount(self, obj):
-        return len(self.get_unreachable_assets(obj))
-
-    def get_reachable_amount(self, obj):
-        return len(self.get_reachable_assets(obj))
-
-    @staticmethod
-    def get_assets_amount(obj):
-        return len(obj.assets)
+    @classmethod
+    def setup_eager_loading(cls, queryset):
+        """ Perform necessary eager loading of data. """
+        queryset = queryset.prefetch_related('cmd_filters', 'nodes')
+        return queryset
 
 
-class SystemUserAuthSerializer(serializers.ModelSerializer):
+class SystemUserAuthSerializer(AuthSerializer):
     """
     系统用户认证信息
     """
-    password = serializers.CharField(max_length=1024)
-    private_key = serializers.CharField(max_length=4096)
 
     class Meta:
         model = SystemUser
         fields = [
             "id", "name", "username", "protocol",
-            "password", "private_key",
+            "login_mode", "password", "private_key",
         ]
 
 
-class AssetSystemUserSerializer(serializers.ModelSerializer):
-    """
-    查看授权的资产系统用户的数据结构，这个和AssetSerializer不同，字段少
-    """
-    class Meta:
-        model = SystemUser
-        fields = ('id', 'name', 'username', 'priority', 'protocol',  'comment',)
-
 
 class SystemUserSimpleSerializer(serializers.ModelSerializer):
     """
@@ -66,4 +59,7 @@ class SystemUserSimpleSerializer(serializers.ModelSerializer):
     """
     class Meta:
         model = SystemUser
-        fields = ('id', 'name', 'username')
+        fields = ('id', 'name', 'username')
+
+
+

apps/assets/signals.py

@@ -1,5 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-from django.dispatch import Signal
-
-on_app_ready = Signal()

apps/assets/signals_handler.py

@@ -1,14 +1,17 @@
 # -*- coding: utf-8 -*-
 #
-
-from django.db.models.signals import post_save, m2m_changed
+from collections import defaultdict
+from django.db.models.signals import post_save, m2m_changed, post_delete
 from django.dispatch import receiver
 
 from common.utils import get_logger
-from .models import Asset, SystemUser, Node
-from .tasks import update_assets_hardware_info_util, \
-    test_asset_connectability_util, push_system_user_to_node, \
-    push_node_system_users_to_asset
+from common.decorator import on_transaction_commit
+from .models import Asset, SystemUser, Node, AuthBook
+from .tasks import (
+    update_assets_hardware_info_util,
+    test_asset_connectivity_util,
+    push_system_user_to_assets
+)
 
 
 logger = get_logger(__file__)
@@ -20,52 +23,99 @@ def update_asset_hardware_info_on_created(asset):
 
 
 def test_asset_conn_on_created(asset):
-    logger.debug("Test asset `{}` connectability".format(asset))
-    test_asset_connectability_util.delay([asset])
-
-
-def set_asset_root_node(asset):
-    logger.debug("Set asset default node: {}".format(Node.root()))
-    asset.nodes.add(Node.root())
+    logger.debug("Test asset `{}` connectivity".format(asset))
+    test_asset_connectivity_util.delay([asset])
 
 
 @receiver(post_save, sender=Asset, dispatch_uid="my_unique_identifier")
+@on_transaction_commit
 def on_asset_created_or_update(sender, instance=None, created=False, **kwargs):
-    set_asset_root_node(instance)
     if created:
         logger.info("Asset `{}` create signal received".format(instance))
+
+        # 获取资产硬件信息
         update_asset_hardware_info_on_created(instance)
         test_asset_conn_on_created(instance)
 
+        # 过期节点资产数量
+        nodes = instance.nodes.all()
+        Node.expire_nodes_assets_amount(nodes)
+
+
+@receiver(post_delete, sender=Asset, dispatch_uid="my_unique_identifier")
+def on_asset_delete(sender, instance=None, **kwargs):
+    # 过期节点资产数量
+    nodes = instance.nodes.all()
+    Node.expire_nodes_assets_amount(nodes)
+
 
 @receiver(post_save, sender=SystemUser, dispatch_uid="my_unique_identifier")
 def on_system_user_update(sender, instance=None, created=True, **kwargs):
     if instance and not created:
-        for node in instance.nodes.all():
-            push_system_user_to_node(instance, node)
+        logger.info("System user `{}` update signal received".format(instance))
+        assets = instance.assets.all()
+        push_system_user_to_assets.delay(instance, assets)
 
 
 @receiver(m2m_changed, sender=SystemUser.nodes.through)
-def on_system_user_node_change(sender, instance=None, **kwargs):
+def on_system_user_nodes_change(sender, instance=None, **kwargs):
     if instance and kwargs["action"] == "post_add":
-        for pk in kwargs['pk_set']:
-            node = kwargs['model'].objects.get(pk=pk)
-            push_system_user_to_node(instance, node)
+        logger.info("System user `{}` nodes update signal received".format(instance))
+        assets = set()
+        nodes = kwargs['model'].objects.filter(pk__in=kwargs['pk_set'])
+        for node in nodes:
+            assets.update(set(node.get_all_assets()))
+        instance.assets.add(*tuple(assets))
+
+
+@receiver(m2m_changed, sender=SystemUser.assets.through)
+def on_system_user_assets_change(sender, instance=None, **kwargs):
+    if instance and kwargs["action"] == "post_add":
+        assets = kwargs['model'].objects.filter(pk__in=kwargs['pk_set'])
+        push_system_user_to_assets.delay(instance, assets)
 
 
 @receiver(m2m_changed, sender=Asset.nodes.through)
 def on_asset_node_changed(sender, instance=None, **kwargs):
-    if isinstance(instance, Asset) and kwargs['action'] == 'post_add':
-        logger.debug("Asset node change signal received")
-        for pk in kwargs['pk_set']:
-            node = kwargs['model'].objects.get(pk=pk)
-            push_node_system_users_to_asset(node, [instance])
+    logger.debug("Asset nodes change signal received")
+    if isinstance(instance, Asset):
+        if kwargs['action'] == 'pre_remove':
+            nodes = kwargs['model'].objects.filter(pk__in=kwargs['pk_set'])
+            Node.expire_nodes_assets_amount(nodes)
+        if kwargs['action'] == 'post_add':
+            nodes = kwargs['model'].objects.filter(pk__in=kwargs['pk_set'])
+            Node.expire_nodes_assets_amount(nodes)
+            system_users_assets = defaultdict(set)
+            system_users = SystemUser.objects.filter(nodes__in=nodes)
+            # 清理节点缓存
+            for system_user in system_users:
+                system_users_assets[system_user].update({instance})
+            for system_user, assets in system_users_assets.items():
+                system_user.assets.add(*tuple(assets))
 
 
 @receiver(m2m_changed, sender=Asset.nodes.through)
 def on_node_assets_changed(sender, instance=None, **kwargs):
-    if isinstance(instance, Node) and kwargs['action'] == 'post_add':
-        logger.debug("Node assets change signal received")
+    if isinstance(instance, Node):
+        logger.debug("Node assets change signal {} received".format(instance))
+        # 当节点和资产关系发生改变时，过期资产数量缓存
+        instance.expire_assets_amount()
         assets = kwargs['model'].objects.filter(pk__in=kwargs['pk_set'])
-        push_node_system_users_to_asset(instance, assets)
+        if kwargs['action'] == 'post_add':
+            # 重新关联系统用户和资产的关系
+            system_users = SystemUser.objects.filter(nodes=instance)
+            for system_user in system_users:
+                system_user.assets.add(*tuple(assets))
 
+
+@receiver(post_save, sender=Node)
+def on_node_update_or_created(sender, instance=None, created=False, **kwargs):
+    if instance and not created:
+        instance.expire_full_value()
+
+
+@receiver(post_save, sender=AuthBook)
+def on_auth_book_created(sender, instance=None, created=False, **kwargs):
+    if created:
+        logger.debug('Receive create auth book object signal.')
+        instance.set_version_and_latest()

apps/assets/tasks.py

@@ -3,50 +3,88 @@ import json
 import re
 import os
 
+from collections import defaultdict
 from celery import shared_task
-from django.core.cache import cache
 from django.utils.translation import ugettext as _
+from django.core.cache import cache
 
-from common.utils import get_object_or_none, capacity_convert, \
-    sum_capacity, encrypt_password, get_logger
-from common.celery import register_as_period_task, after_app_shutdown_clean, \
-    after_app_ready_start, app as celery_app
+from common.utils import (
+    capacity_convert, sum_capacity, encrypt_password, get_logger
+)
+from ops.celery.decorator import (
+    register_as_period_task, after_app_shutdown_clean_periodic
+)
 
-from .models import SystemUser, AdminUser, Asset, Cluster
+from .models import SystemUser, AdminUser
+from .models.utils import Connectivity
 from . import const
 
 
 FORKS = 10
 TIMEOUT = 60
 logger = get_logger(__file__)
-CACHE_MAX_TIME = 60*60*60
+CACHE_MAX_TIME = 60*60*2
 disk_pattern = re.compile(r'^hd|sd|xvd|vd')
 PERIOD_TASK = os.environ.get("PERIOD_TASK", "on")
 
 
+def check_asset_can_run_ansible(asset):
+    if not asset.is_active:
+        msg = _("Asset has been disabled, skipped: {}").format(asset)
+        logger.info(msg)
+        return False
+    if not asset.is_support_ansible():
+        msg = _("Asset may not be support ansible, skipped: {}").format(asset)
+        logger.info(msg)
+        return False
+    return True
+
+
+def clean_hosts(assets):
+    clean_assets = []
+    for asset in assets:
+        if not check_asset_can_run_ansible(asset):
+            continue
+        clean_assets.append(asset)
+    if not clean_assets:
+        logger.info(_("No assets matched, stop task"))
+    return clean_assets
+
+
+def clean_hosts_by_protocol(system_user, assets):
+    hosts = [
+        asset for asset in assets
+        if asset.has_protocol(system_user.protocol)
+    ]
+    if not hosts:
+        msg = _("No assets matched related system user protocol, stop task")
+        logger.info(msg)
+    return hosts
+
+
 @shared_task
-def set_assets_hardware_info(result, **kwargs):
+def set_assets_hardware_info(assets, result, **kwargs):
     """
     Using ops task run result, to update asset info
 
     @shared_task must be exit, because we using it as a task callback, is must
     be a celery task also
+    :param assets:
     :param result:
     :param kwargs: {task_name: ""}
     :return:
     """
     result_raw = result[0]
     assets_updated = []
-    for hostname, info in result_raw.get('ok', {}).items():
+    success_result = result_raw.get('ok', {})
+
+    for asset in assets:
+        hostname = asset.hostname
+        info = success_result.get(hostname, {})
         info = info.get('setup', {}).get('ansible_facts', {})
         if not info:
-            logger.error("Get asset info failed: {}".format(hostname))
+            logger.error(_("Get asset info failed: {}").format(hostname))
             continue
-
-        asset = get_object_or_none(Asset, hostname=hostname)
-        if not asset:
-            continue
-
         ___vendor = info.get('ansible_system_vendor', 'Unknown')
         ___model = info.get('ansible_product_name', 'Unknown')
         ___sn = info.get('ansible_product_serial', 'Unknown')
@@ -58,8 +96,12 @@ def set_assets_hardware_info(result, **kwargs):
             ___cpu_model = 'Unknown'
         ___cpu_model = ___cpu_model[:64]
         ___cpu_count = info.get('ansible_processor_count', 0)
-        ___cpu_cores = info.get('ansible_processor_cores', None) or len(info.get('ansible_processor', []))
-        ___memory = '%s %s' % capacity_convert('{} MB'.format(info.get('ansible_memtotal_mb')))
+        ___cpu_cores = info.get('ansible_processor_cores', None) or \
+                       len(info.get('ansible_processor', []))
+        ___cpu_vcpus = info.get('ansible_processor_vcpus', 0)
+        ___memory = '%s %s' % capacity_convert(
+            '{} MB'.format(info.get('ansible_memtotal_mb'))
+        )
         disk_info = {}
         for dev, dev_info in info.get('ansible_devices', {}).items():
             if disk_pattern.match(dev) and dev_info['removable'] == '0':
@@ -67,7 +109,7 @@ def set_assets_hardware_info(result, **kwargs):
         ___disk_total = '%s %s' % sum_capacity(disk_info.values())
         ___disk_info = json.dumps(disk_info)
 
-        ___platform = info.get('ansible_system', 'Unknown')
+        # ___platform = info.get('ansible_system', 'Unknown')
         ___os = info.get('ansible_distribution', 'Unknown')
         ___os_version = info.get('ansible_distribution_version', 'Unknown')
         ___os_arch = info.get('ansible_architecture', 'Unknown')
@@ -91,32 +133,30 @@ def update_assets_hardware_info_util(assets, task_name=None):
     """
     from ops.utils import update_or_create_ansible_task
     if task_name is None:
-        # task_name = _("Update some assets hardware info")
-        task_name = _("更新资产硬件信息")
+        task_name = _("Update some assets hardware info")
     tasks = const.UPDATE_ASSETS_HARDWARE_TASKS
-    hostname_list = [asset.hostname for asset in assets if asset.is_active and asset.is_unixlike()]
+    hosts = clean_hosts(assets)
+    if not hosts:
+        return {}
+    created_by = str(assets[0].org_id)
     task, created = update_or_create_ansible_task(
-        task_name, hosts=hostname_list, tasks=tasks, pattern='all',
-        options=const.TASK_OPTIONS, run_as_admin=True, created_by='System',
+        task_name, hosts=hosts, tasks=tasks, created_by=created_by,
+        pattern='all', options=const.TASK_OPTIONS, run_as_admin=True,
     )
     result = task.run()
-    # Todo: may be somewhere using
-    # Manual run callback function
-    set_assets_hardware_info(result)
+    set_assets_hardware_info(assets, result)
     return result
 
 
 @shared_task
 def update_asset_hardware_info_manual(asset):
-    # task_name = _("Update asset hardware info")
-    task_name = _("更新资产硬件信息")
-    return update_assets_hardware_info_util([asset], task_name=task_name)
+    task_name = _("Update asset hardware info: {}").format(asset.hostname)
+    update_assets_hardware_info_util(
+        [asset], task_name=task_name
+    )
 
 
-@celery_app.task
-@register_as_period_task(interval=3600)
-@after_app_ready_start
-@after_app_shutdown_clean
+@shared_task
 def update_assets_hardware_info_period():
     """
     Update asset hardware period task
@@ -126,126 +166,71 @@ def update_assets_hardware_info_period():
         logger.debug("Period task disabled, update assets hardware info pass")
         return
 
-    from ops.utils import update_or_create_ansible_task
-    # task_name = _("Update assets hardware info period")
-    task_name = _("定期更新资产硬件信息")
-    hostname_list = [
-        asset.hostname for asset in Asset.objects.all()
-        if asset.is_active and asset.is_unixlike()
-    ]
-    tasks = const.UPDATE_ASSETS_HARDWARE_TASKS
-
-    # Only create, schedule by celery beat
-    update_or_create_ansible_task(
-        task_name, hosts=hostname_list, tasks=tasks, pattern='all',
-        options=const.TASK_OPTIONS, run_as_admin=True, created_by='System',
-        interval=60*60*24, is_periodic=True, callback=set_assets_hardware_info.name,
-    )
-
 
 ##  ADMIN USER CONNECTIVE  ##
 
-@shared_task
-def set_admin_user_connectability_info(result, **kwargs):
-    admin_user = kwargs.get("admin_user")
-    task_name = kwargs.get("task_name")
-    if admin_user is None and task_name is not None:
-        admin_user = task_name.split(":")[-1]
-
-    raw, summary = result
-    cache_key = const.ADMIN_USER_CONN_CACHE_KEY.format(admin_user)
-    cache.set(cache_key, summary, CACHE_MAX_TIME)
-
-    for i in summary.get('contacted', []):
-        asset_conn_cache_key = const.ASSET_ADMIN_CONN_CACHE_KEY.format(i)
-        cache.set(asset_conn_cache_key, 1, CACHE_MAX_TIME)
-
-    for i, msg in summary.get('dark', {}).items():
-        asset_conn_cache_key = const.ASSET_ADMIN_CONN_CACHE_KEY.format(i)
-        cache.set(asset_conn_cache_key, 0, CACHE_MAX_TIME)
-        logger.error(msg)
-
 
 @shared_task
-def test_admin_user_connectability_util(admin_user, task_name):
-    """
-    Test asset admin user can connect or not. Using ansible api do that
-    :param admin_user:
-    :param task_name:
-    :return:
-    """
-    from ops.utils import update_or_create_ansible_task
-
-    assets = admin_user.get_related_assets()
-    hosts = [asset.hostname for asset in assets
-             if asset.is_active and asset.is_unixlike()]
-    if not hosts:
-        return
-    tasks = const.TEST_ADMIN_USER_CONN_TASKS
-    task, created = update_or_create_ansible_task(
-        task_name=task_name, hosts=hosts, tasks=tasks, pattern='all',
-        options=const.TASK_OPTIONS, run_as_admin=True, created_by='System',
-    )
-    result = task.run()
-    set_admin_user_connectability_info(result, admin_user=admin_user.name)
-    return result
-
-
-@celery_app.task
-@register_as_period_task(interval=3600)
-@after_app_ready_start
-@after_app_shutdown_clean
-def test_admin_user_connectability_period():
-    """
-    A period task that update the ansible task period
-    """
-    if PERIOD_TASK != "on":
-        logger.debug("Period task disabled, test admin user connectability pass")
-        return
-
-    admin_users = AdminUser.objects.all()
-    for admin_user in admin_users:
-        # task_name = _("Test admin user connectability period: {}".format(admin_user.name))
-        task_name = _("定期测试管理账号可连接性: {}".format(admin_user.name))
-        test_admin_user_connectability_util(admin_user, task_name)
-
-
-@shared_task
-def test_admin_user_connectability_manual(admin_user):
-    # task_name = _("Test admin user connectability: {}").format(admin_user.name)
-    task_name = _("测试管理行号可连接性: {}").format(admin_user.name)
-    return test_admin_user_connectability_util.delay(admin_user, task_name)
-
-
-@shared_task
-def test_asset_connectability_util(assets, task_name=None):
+def test_asset_connectivity_util(assets, task_name=None):
     from ops.utils import update_or_create_ansible_task
 
     if task_name is None:
-        # task_name = _("Test assets connectability")
-        task_name = _("测试资产可连接性")
-    hosts = [asset.hostname for asset in assets if asset.is_active and asset.is_unixlike()]
-    if not hosts:
-        logger.info("No hosts, passed")
-        return {}
-    tasks = const.TEST_ADMIN_USER_CONN_TASKS
-    task, created = update_or_create_ansible_task(
-        task_name=task_name, hosts=hosts, tasks=tasks, pattern='all',
-        options=const.TASK_OPTIONS, run_as_admin=True, created_by='System',
-    )
-    result = task.run()
-    summary = result[1]
-    for k in summary.get('dark'):
-        cache.set(const.ASSET_ADMIN_CONN_CACHE_KEY.format(k), 0, CACHE_MAX_TIME)
+        task_name = _("Test assets connectivity")
 
-    for k in summary.get('contacted'):
-        cache.set(const.ASSET_ADMIN_CONN_CACHE_KEY.format(k), 1, CACHE_MAX_TIME)
-    return summary
+    hosts = clean_hosts(assets)
+    if not hosts:
+        return {}
+
+    hosts_category = {
+        'linux': {
+            'hosts': [],
+            'tasks': const.TEST_ADMIN_USER_CONN_TASKS
+        },
+        'windows': {
+            'hosts': [],
+            'tasks': const.TEST_WINDOWS_ADMIN_USER_CONN_TASKS
+        }
+    }
+    for host in hosts:
+        hosts_list = hosts_category['windows']['hosts'] if host.is_windows() \
+            else hosts_category['linux']['hosts']
+        hosts_list.append(host)
+
+    results_summary = dict(
+        contacted=defaultdict(dict), dark=defaultdict(dict), success=True
+    )
+    created_by = assets[0].org_id
+    for k, value in hosts_category.items():
+        if not value['hosts']:
+            continue
+        task, created = update_or_create_ansible_task(
+            task_name=task_name, hosts=value['hosts'], tasks=value['tasks'],
+            pattern='all', options=const.TASK_OPTIONS, run_as_admin=True,
+            created_by=created_by,
+        )
+        raw, summary = task.run()
+        success = summary.get('success', False)
+        contacted = summary.get('contacted', {})
+        dark = summary.get('dark', {})
+
+        results_summary['success'] &= success
+        results_summary['contacted'].update(contacted)
+        results_summary['dark'].update(dark)
+
+    for asset in assets:
+        if asset.hostname in results_summary.get('dark', {}).keys():
+            asset.connectivity = Connectivity.unreachable()
+        elif asset.hostname in results_summary.get('contacted', {}).keys():
+            asset.connectivity = Connectivity.reachable()
+        else:
+            asset.connectivity = Connectivity.unknown()
+    return results_summary
 
 
 @shared_task
-def test_asset_connectability_manual(asset):
-    summary = test_asset_connectability_util([asset])
+def test_asset_connectivity_manual(asset):
+    task_name = _("Test assets connectivity: {}").format(asset)
+    summary = test_asset_connectivity_util([asset], task_name=task_name)
 
     if summary.get('dark'):
         return False, summary['dark']
@@ -253,73 +238,142 @@ def test_asset_connectability_manual(asset):
         return True, ""
 
 
-##  System user connective ##
-
 @shared_task
-def set_system_user_connectablity_info(result, **kwargs):
-    summary = result[1]
-    task_name = kwargs.get("task_name")
-    system_user = kwargs.get("system_user")
-    if system_user is None:
-        system_user = task_name.split(":")[-1]
-    cache_key = const.SYSTEM_USER_CONN_CACHE_KEY.format(system_user)
-    cache.set(cache_key, summary, CACHE_MAX_TIME)
-
-
-@shared_task
-def test_system_user_connectability_util(system_user, task_name):
+def test_admin_user_connectivity_util(admin_user, task_name):
     """
-    Test system cant connect his assets or not.
-    :param system_user:
+    Test asset admin user can connect or not. Using ansible api do that
+    :param admin_user:
     :param task_name:
     :return:
     """
-    from ops.utils import update_or_create_ansible_task
-    assets = system_user.assets
-    hosts = [asset.hostname for asset in assets if asset.is_active and asset.is_unixlike()]
-    tasks = const.TEST_SYSTEM_USER_CONN_TASKS
+    assets = admin_user.get_related_assets()
+    hosts = clean_hosts(assets)
     if not hosts:
-        logger.info("No hosts, passed")
         return {}
-    task, created = update_or_create_ansible_task(
-        task_name, hosts=hosts, tasks=tasks, pattern='all',
-        options=const.TASK_OPTIONS,
-        run_as=system_user.name, created_by="System",
-    )
-    result = task.run()
-    set_system_user_connectablity_info(result, system_user=system_user.name)
-    return result
-
-
-@shared_task
-def test_system_user_connectability_manual(system_user):
-    task_name = _("Test system user connectability: {}").format(system_user)
-    return test_system_user_connectability_util(system_user, task_name)
+    summary = test_asset_connectivity_util(hosts, task_name)
+    return summary
 
 
 @shared_task
 @register_as_period_task(interval=3600)
-@after_app_ready_start
-@after_app_shutdown_clean
-def test_system_user_connectability_period():
+def test_admin_user_connectivity_period():
+    """
+    A period task that update the ansible task period
+    """
     if PERIOD_TASK != "on":
-        logger.debug("Period task disabled, test system user connectability pass")
+        logger.debug('Period task off, skip')
         return
+    key = '_JMS_TEST_ADMIN_USER_CONNECTIVITY_PERIOD'
+    prev_execute_time = cache.get(key)
+    if prev_execute_time:
+        logger.debug("Test admin user connectivity, less than 40 minutes, skip")
+        return
+    cache.set(key, 1, 60*40)
+    admin_users = AdminUser.objects.all()
+    for admin_user in admin_users:
+        task_name = _("Test admin user connectivity period: {}").format(admin_user.name)
+        test_admin_user_connectivity_util(admin_user, task_name)
+    cache.set(key, 1, 60*40)
 
+
+@shared_task
+def test_admin_user_connectivity_manual(admin_user):
+    task_name = _("Test admin user connectivity: {}").format(admin_user.name)
+    test_admin_user_connectivity_util(admin_user, task_name)
+    return True
+
+
+##  System user connective ##
+
+
+@shared_task
+def test_system_user_connectivity_util(system_user, assets, task_name):
+    """
+    Test system cant connect his assets or not.
+    :param system_user:
+    :param assets:
+    :param task_name:
+    :return:
+    """
+    from ops.utils import update_or_create_ansible_task
+
+    hosts = clean_hosts(assets)
+    if not hosts:
+        return {}
+
+    hosts = clean_hosts_by_protocol(system_user, hosts)
+    if not hosts:
+        return {}
+
+    hosts_category = {
+        'linux': {
+            'hosts': [],
+            'tasks': const.TEST_SYSTEM_USER_CONN_TASKS
+        },
+        'windows': {
+            'hosts': [],
+            'tasks': const.TEST_WINDOWS_SYSTEM_USER_CONN_TASKS
+        }
+    }
+    for host in hosts:
+        hosts_list = hosts_category['windows']['hosts'] if host.is_windows() \
+            else hosts_category['linux']['hosts']
+        hosts_list.append(host)
+
+    results_summary = dict(
+        contacted=defaultdict(dict), dark=defaultdict(dict), success=True
+    )
+    for k, value in hosts_category.items():
+        if not value['hosts']:
+            continue
+        task, created = update_or_create_ansible_task(
+            task_name=task_name, hosts=value['hosts'], tasks=value['tasks'],
+            pattern='all', options=const.TASK_OPTIONS,
+            run_as=system_user.username, created_by=system_user.org_id,
+        )
+        raw, summary = task.run()
+        success = summary.get('success', False)
+        contacted = summary.get('contacted', {})
+        dark = summary.get('dark', {})
+
+        results_summary['success'] &= success
+        results_summary['contacted'].update(contacted)
+        results_summary['dark'].update(dark)
+
+    system_user.set_connectivity(results_summary)
+    return results_summary
+
+
+@shared_task
+def test_system_user_connectivity_manual(system_user):
+    task_name = _("Test system user connectivity: {}").format(system_user)
+    assets = system_user.get_related_assets()
+    return test_system_user_connectivity_util(system_user, assets, task_name)
+
+
+@shared_task
+def test_system_user_connectivity_a_asset(system_user, asset):
+    task_name = _("Test system user connectivity: {} => {}").format(
+        system_user, asset
+    )
+    return test_system_user_connectivity_util(system_user, [asset], task_name)
+
+
+@shared_task
+def test_system_user_connectivity_period():
+    if PERIOD_TASK != "on":
+        logger.debug("Period task disabled, test system user connectivity pass")
+        return
     system_users = SystemUser.objects.all()
     for system_user in system_users:
-        # task_name = _("Test system user connectability period: {}".format(system_user))
-        task_name = _("定期测试系统用户可连接性: {}".format(system_user))
-        test_system_user_connectability_util(system_user, task_name)
+        task_name = _("Test system user connectivity period: {}").format(system_user)
+        assets = system_user.get_related_assets()
+        test_system_user_connectivity_util(system_user, assets, task_name)
 
 
 ####  Push system user tasks ####
 
-def get_push_system_user_tasks(system_user):
-    # Set root as system user is dangerous
-    if system_user.username == "root":
-        return []
-
+def get_push_linux_system_user_tasks(system_user):
     tasks = []
     if system_user.password:
         tasks.append({
@@ -332,6 +386,24 @@ def get_push_system_user_tasks(system_user):
                 ),
             }
         })
+        tasks.extend([
+            {
+                'name': 'Check home dir exists',
+                'action': {
+                    'module': 'stat',
+                    'args': 'path=/home/{}'.format(system_user.username)
+                },
+                'register': 'home_existed'
+            },
+            {
+                'name': "Set home dir permission",
+                'action': {
+                    'module': 'file',
+                    'args': "path=/home/{0} owner={0} group={0} mode=700".format(system_user.username)
+                },
+                'when': 'home_existed.stat.exists == true'
+            }
+        ])
     if system_user.public_key:
         tasks.append({
             'name': 'Set {} authorized key'.format(system_user.username),
@@ -343,6 +415,12 @@ def get_push_system_user_tasks(system_user):
             }
         })
     if system_user.sudo:
+        sudo = system_user.sudo.replace('\r\n', '\n').replace('\r', '\n')
+        sudo_list = sudo.split('\n')
+        sudo_tmp = []
+        for s in sudo_list:
+            sudo_tmp.append(s.strip(','))
+        sudo = ','.join(sudo_tmp)
         tasks.append({
             'name': 'Set {} sudo setting'.format(system_user.username),
             'action': {
@@ -350,91 +428,190 @@ def get_push_system_user_tasks(system_user):
                 'args': "dest=/etc/sudoers state=present regexp='^{0} ALL=' "
                         "line='{0} ALL=(ALL) NOPASSWD: {1}' "
                         "validate='visudo -cf %s'".format(
-                    system_user.username,
-                    system_user.sudo,
+                    system_user.username, sudo,
                 )
             }
         })
+
+    return tasks
+
+
+def get_push_windows_system_user_tasks(system_user):
+    tasks = []
+    if system_user.password:
+        tasks.append({
+            'name': 'Add user {}'.format(system_user.username),
+            'action': {
+                'module': 'win_user',
+                'args': 'fullname={} '
+                        'name={} '
+                        'password={} '
+                        'state=present '
+                        'update_password=always '
+                        'password_expired=no '
+                        'password_never_expires=yes '
+                        'groups="Users,Remote Desktop Users" '
+                        'groups_action=add '
+                        ''.format(system_user.name,
+                                  system_user.username,
+                                  system_user.password),
+            }
+        })
     return tasks
 
 
-@shared_task
-def push_system_user_util(system_users, assets, task_name):
-    from ops.utils import update_or_create_ansible_task
-    tasks = []
-    for system_user in system_users:
-        if not system_user.is_need_push():
-            msg = "push system user `{}` passed, may be not auto push or ssh " \
-                  "protocol is not ssh".format(system_user.name)
-            logger.info(msg)
-            continue
-        tasks.extend(get_push_system_user_tasks(system_user))
-
-    if not tasks:
-        logger.info("Not tasks, passed")
-        return {}
-
-    hosts = [asset.hostname for asset in assets if asset.is_active and asset.is_unixlike()]
-    if not hosts:
-        logger.info("Not hosts, passed")
-        return {}
-    task, created = update_or_create_ansible_task(
-        task_name=task_name, hosts=hosts, tasks=tasks, pattern='all',
-        options=const.TASK_OPTIONS, run_as_admin=True, created_by='System'
-    )
-    return task.run()
-
-
-def get_node_push_system_user_task_name(system_user, node):
-
-    # return _("Push system user to node: {} => {}").format(
-    return _("推送系统用户到节点资产: {} => {}").format(
-        system_user.name,
-        node.value
-    )
-
-
-def push_system_user_to_node(system_user, node):
-    assets = node.get_all_assets()
-    task_name = get_node_push_system_user_task_name(system_user, node)
-    push_system_user_util.delay([system_user], assets, task_name)
-
-
-@shared_task
-def push_system_user_related_nodes(system_user):
-    if not system_user.is_need_push():
-        msg = "push system user `{}` passed, may be not auto push or ssh " \
-              "protocol is not ssh".format(system_user.name)
+def get_push_system_user_tasks(host, system_user):
+    if host.is_unixlike():
+        tasks = get_push_linux_system_user_tasks(system_user)
+    elif host.is_windows():
+        tasks = get_push_windows_system_user_tasks(system_user)
+    else:
+        msg = _(
+            "The asset {} system platform {} does not "
+            "support run Ansible tasks".format(host.hostname, host.platform)
+        )
         logger.info(msg)
-        return
+        tasks = []
+    return tasks
 
-    nodes = system_user.nodes.all()
-    for node in nodes:
-        push_system_user_to_node(system_user, node)
+
+@shared_task
+def push_system_user_util(system_user, assets, task_name):
+    from ops.utils import update_or_create_ansible_task
+    if not system_user.is_need_push():
+        msg = _("Push system user task skip, auto push not enable or "
+                "protocol is not ssh or rdp: {}").format(system_user.name)
+        logger.info(msg)
+        return {}
+
+    # Set root as system user is dangerous
+    if system_user.username.lower() in ["root", "administrator"]:
+        msg = _("For security, do not push user {}".format(system_user.username))
+        logger.info(msg)
+        return {}
+
+    hosts = clean_hosts(assets)
+    if not hosts:
+        return {}
+
+    hosts = clean_hosts_by_protocol(system_user, hosts)
+    if not hosts:
+        return {}
+
+    for host in hosts:
+        system_user.load_specific_asset_auth(host)
+        tasks = get_push_system_user_tasks(host, system_user)
+        if not tasks:
+            continue
+        task, created = update_or_create_ansible_task(
+            task_name=task_name, hosts=[host], tasks=tasks, pattern='all',
+            options=const.TASK_OPTIONS, run_as_admin=True,
+            created_by=system_user.org_id,
+        )
+        task.run()
 
 
 @shared_task
 def push_system_user_to_assets_manual(system_user):
-    push_system_user_related_nodes(system_user)
+    assets = system_user.get_related_assets()
+    task_name = _("Push system users to assets: {}").format(system_user.name)
+    return push_system_user_util(system_user, assets, task_name=task_name)
 
 
-def push_node_system_users_to_asset(node, assets):
-    system_users = []
-    nodes = node.ancestor_with_node
-    # 获取该节点所有父节点有的系统用户, 然后推送
-    for n in nodes:
-        system_users.extend(list(n.systemuser_set.all()))
+@shared_task
+def push_system_user_a_asset_manual(system_user, asset):
+    task_name = _("Push system users to asset: {} => {}").format(
+        system_user.name, asset
+    )
+    return push_system_user_util(system_user, [asset], task_name=task_name)
 
-    if system_users:
-        # task_name = _("Push system users to node: {}").format(node.value)
-        task_name = _("推送节点系统用户到新加入资产中: {}").format(node.value)
-        push_system_user_util.delay(system_users, assets, task_name)
+
+@shared_task
+def push_system_user_to_assets(system_user, assets):
+    task_name = _("Push system users to assets: {}").format(system_user.name)
+    return push_system_user_util(system_user, assets, task_name)
+
+
+@shared_task
+@after_app_shutdown_clean_periodic
+def test_system_user_connectability_period():
+    pass
+
+
+@shared_task
+@after_app_shutdown_clean_periodic
+def test_admin_user_connectability_period():
+    pass
+
+
+#### Test Asset user connectivity task ####
+
+def get_test_asset_user_connectivity_tasks(asset):
+    if asset.is_unixlike():
+        tasks = const.TEST_ASSET_USER_CONN_TASKS
+    elif asset.is_windows():
+        tasks = const.TEST_WINDOWS_ASSET_USER_CONN_TASKS
+    else:
+        msg = _(
+            "The asset {} system platform {} does not "
+            "support run Ansible tasks".format(asset.hostname, asset.platform)
+        )
+        logger.info(msg)
+        tasks = []
+    return tasks
+
+
+@shared_task
+def test_asset_user_connectivity_util(asset_user, task_name, run_as_admin=False):
+    """
+    :param asset_user: <AuthBook>对象
+    :param task_name:
+    :param run_as_admin:
+    :return:
+    """
+    from ops.utils import update_or_create_ansible_task
+
+    if not check_asset_can_run_ansible(asset_user.asset):
+        return
+
+    tasks = get_test_asset_user_connectivity_tasks(asset_user.asset)
+    if not tasks:
+        logger.debug("No tasks ")
+        return
+
+    args = (task_name,)
+    kwargs = {
+        'hosts': [asset_user.asset], 'tasks': tasks,
+        'pattern': 'all', 'options': const.TASK_OPTIONS,
+        'created_by': asset_user.org_id,
+    }
+    if run_as_admin:
+        kwargs["run_as_admin"] = True
+    else:
+        kwargs["run_as"] = asset_user.username
+    task, created = update_or_create_ansible_task(*args, **kwargs)
+    raw, summary = task.run()
+    asset_user.set_connectivity(summary)
+
+
+@shared_task
+def test_asset_users_connectivity_manual(asset_users, run_as_admin=False):
+    """
+    :param asset_users: <AuthBook>对象
+    """
+    for asset_user in asset_users:
+        task_name = _("Test asset user connectivity: {}").format(asset_user)
+        test_asset_user_connectivity_util(asset_user, task_name, run_as_admin=run_as_admin)
 
 
 # @shared_task
 # @register_as_period_task(interval=3600)
 # @after_app_ready_start
-# # @after_app_shutdown_clean
+# @after_app_shutdown_clean_periodic
 # def push_system_user_period():
 #     for system_user in SystemUser.objects.all():
 #         push_system_user_related_nodes(system_user)
+
+
+
+

apps/assets/templates/assets/_admin_user_import_modal.html

@@ -0,0 +1,6 @@
+{% extends '_import_modal.html' %}
+{% load i18n %}
+
+{% block modal_title%}{% trans "Import admin user" %}{% endblock %}
+
+{% block import_modal_download_template_url %}{% url "api-assets:admin-user-list" %}{% endblock %}