fix: 修改获取org时，thread_local缓存问题

Dev

README.md

@@ -4,9 +4,117 @@
 [![Django](https://img.shields.io/badge/django-2.2-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
 [![Docker Pulls](https://img.shields.io/docker/pulls/jumpserver/jms_all.svg)](https://hub.docker.com/u/jumpserver)
 
-|Developer Wanted|
-|------------------|
-|JumpServer 正在寻找开发者，一起为改变世界做些贡献吧，哪怕一点点，联系我 <ibuler@fit2cloud.com> |
+- [ENGLISH](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
+
+## 紧急BUG修复通知
+JumpServer发现远程执行漏洞，请速度修复
+
+非常感谢  **reactivity of Alibaba Hackerone bug bounty program**(瑞典) 向我们报告了此 BUG
+
+**影响版本:**
+```
+< v2.6.2
+< v2.5.4
+< v2.4.5 
+= v1.5.9
+>= v1.5.3
+```
+**安全版本:**
+```
+>= v2.6.2
+>= v2.5.4
+>= v2.4.5 
+= v1.5.9 （版本号没变）
+< v1.5.3
+```
+
+**修复方案:**
+
+将JumpServer升级至安全版本；
+
+**临时修复方案:**
+
+修改 Nginx 配置文件屏蔽漏洞接口 
+
+```
+/api/v1/authentication/connection-token/
+/api/v1/users/connection-token/
+```
+
+Nginx 配置文件位置
+```
+# 社区老版本
+/etc/nginx/conf.d/jumpserver.conf
+
+# 企业老版本
+jumpserver-release/nginx/http_server.conf
+ 
+# 新版本在 
+jumpserver-release/compose/config_static/http_server.conf
+```
+
+修改 Nginx 配置文件实例
+```
+### 保证在 /api 之前 和 / 之前
+location /api/v1/authentication/connection-token/ {
+   return 403;
+}
+ 
+location /api/v1/users/connection-token/ {
+   return 403;
+}
+### 新增以上这些
+ 
+location /api/ {
+    proxy_set_header X-Real-IP $remote_addr;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_pass http://core:8080;
+  }
+ 
+...
+```
+
+修改完成后重启 nginx
+
+```
+docker方式: 
+docker restart jms_nginx
+
+nginx方式:
+systemctl restart nginx
+
+```
+
+**修复验证**
+
+```
+$ wget https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_bug_check.sh 
+
+# 使用方法 bash jms_bug_check.sh HOST 
+$ bash jms_bug_check.sh demo.jumpserver.org
+漏洞已修复
+```
+
+**入侵检测**
+
+下载脚本到 jumpserver 日志目录，这个目录中存在 gunicorn.log，然后执行
+
+```
+$ pwd
+/opt/jumpserver/core/logs
+
+$ ls gunicorn.log 
+gunicorn.log
+
+$ wget 'https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_check_attack.sh'
+$ bash jms_check_attack.sh
+系统未被入侵
+```
+
+--------------------------
+
+JumpServer 正在寻找开发者，一起为改变世界做些贡献吧，哪怕一点点，联系我 <ibuler@fit2cloud.com> 
 
 JumpServer 是全球首款开源的堡垒机，使用 GNU GPL v2.0 开源协议，是符合 4A 规范的运维安全审计系统。
 

README_EN.md

@@ -1,22 +1,135 @@
 ## Jumpserver 
 
-![Total visitor](https://visitor-count-badge.herokuapp.com/total.svg?repo_id=jumpserver)
-![Visitors in today](https://visitor-count-badge.herokuapp.com/today.svg?repo_id=jumpserver)
 [![Python3](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)](https://www.python.org/)
-[![Django](https://img.shields.io/badge/django-2.1-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
-[![Ansible](https://img.shields.io/badge/ansible-2.4.2.0-blue.svg?style=plastic)](https://www.ansible.com/)
-[![Paramiko](https://img.shields.io/badge/paramiko-2.4.1-green.svg?style=plastic)](http://www.paramiko.org/)
+[![Django](https://img.shields.io/badge/django-2.2-brightgreen.svg?style=plastic)](https://www.djangoproject.com/)
+[![Docker Pulls](https://img.shields.io/docker/pulls/jumpserver/jms_all.svg)](https://hub.docker.com/u/jumpserver)
 
+---- 
+## CRITICAL BUG WARNING
+
+Recently we have found a critical bug for remote execution vulnerability which leads to pre-auth and info leak, please fix it as soon as possible.
+
+Thanks for **reactivity from Alibaba Hackerone bug bounty program** report us this bug
+
+**Vulnerable version:**
+```
+< v2.6.2
+< v2.5.4
+< v2.4.5 
+= v1.5.9
+>= v1.5.3
+```
+
+**Safe and Stable version:**
+```
+>= v2.6.2
+>= v2.5.4
+>= v2.4.5 
+= v1.5.9 （version tag didn't change）
+< v1.5.3
+```
+
+**Bug Fix Solution:**
+Upgrade to the latest version or the version mentioned above
+
+
+**Temporary Solution (upgrade asap):**
+
+Modify the Nginx config file and disable the vulnerable api listed below
+
+```
+/api/v1/authentication/connection-token/
+/api/v1/users/connection-token/
+```
+
+Path to Nginx config file
+
+```
+# Previous Community version
+/etc/nginx/conf.d/jumpserver.conf
+
+# Previous Enterprise version
+jumpserver-release/nginx/http_server.conf
+ 
+# Latest version
+jumpserver-release/compose/config_static/http_server.conf
+```
+
+Changes in Nginx config file
+
+```
+### Put the following code on top of location server, or before /api and /
+location /api/v1/authentication/connection-token/ {
+   return 403;
+}
+ 
+location /api/v1/users/connection-token/ {
+   return 403;
+}
+### End right here
+ 
+location /api/ {
+    proxy_set_header X-Real-IP $remote_addr;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_pass http://core:8080;
+  }
+ 
+...
+```
+
+Save the file and restart Nginx
+
+```
+docker deployment: 
+$ docker restart jms_nginx
+
+rpm or other deployment:
+$ systemctl restart nginx
+
+```
+
+**Bug Fix Verification**
+
+```
+# Download the following script to check if it is fixed
+$ wget https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_bug_check.sh 
+
+# Run the code to verify it
+$ bash jms_bug_check.sh demo.jumpserver.org
+漏洞已修复 (It means the bug is fixed)
+漏洞未修复 (It means the bug is not fixed and the system is still vulnerable)
+```
+
+
+**Attack Simulation**
+
+Go to the logs directory which should contain gunicorn.log file. Then download the "attack" script and execute it
+
+```
+$ pwd
+/opt/jumpserver/core/logs
+
+$ ls gunicorn.log
+gunicorn.log
+
+$ wget 'https://github.com/jumpserver/jumpserver/releases/download/v2.6.2/jms_check_attack.sh'
+$ bash jms_check_attack.sh
+系统未被入侵 (It means the system is safe)
+系统已被入侵 (It means the system is being attacked)
+```
+
+--------------------------
 
 ----
 
-- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README_EN.md)
+- [中文版](https://github.com/jumpserver/jumpserver/blob/master/README.md)
 
-Jumpserver is the first fully open source bastion in the world, based on the GNU GPL v2.0 open source protocol. Jumpserver is a  professional operation and maintenance audit system conforms to 4A specifications.
+Jumpserver is the world's first open-source PAM (Privileged Access Management System) and is licensed under the GNU GPL v2.0. It is a 4A-compliant professional operation and maintenance security audit system.
 
-Jumpserver is developed using Python / Django, conforms to the Web 2.0 specification, and is equipped with the industry-leading Web Terminal solution which have beautiful interface and great user experience.
+Jumpserver uses Python / Django for development, follows Web 2.0 specifications, and is equipped with an industry-leading Web Terminal solution that provides a beautiful user interface and great user experience
 
-Jumpserver adopts a distributed architecture to support multi-branch deployment across multiple areas. The central node provides APIs, and login nodes are deployed in each branch. It can be scaled horizontally without concurrency restrictions.
+Jumpserver adopts a distributed architecture to support multi-branch deployment across multiple cross-regional areas. The central node provides APIs, and login nodes are deployed in each branch. It can be scaled horizontally without concurrency restrictions.
 
 Change the world, starting from little things.
 
@@ -47,7 +160,7 @@ We provide online demo, demo video and screenshots to get you started quickly.
 We provide the SDK for your other systems to quickly interact with the Jumpserver API.
 
 - [Python](https://github.com/jumpserver/jumpserver-python-sdk) Jumpserver other components use this SDK to complete the interaction.
-- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) 恺珺同学提供的Java版本的SDK thanks to 恺珺 for provide Java SDK
+- [Java](https://github.com/KaiJunYan/jumpserver-java-sdk.git) Thanks to 恺珺 for providing his Java SDK vesrion.
 
 
 ### License & Copyright

apps/applications/api/__init__.py

@@ -1,5 +1,3 @@
 from .application import *
 from .mixin import *
 from .remote_app import *
-from .database_app import *
-from .k8s_app import *

apps/applications/api/application.py

@@ -3,18 +3,17 @@
 
 from orgs.mixins.api import OrgBulkModelViewSet
 
-from .mixin import ApplicationAttrsSerializerViewMixin
 from ..hands import IsOrgAdminOrAppUser
 from .. import models, serializers
 
-__all__ = [
-    'ApplicationViewSet',
-]
+
+__all__ = ['ApplicationViewSet']
 
 
-class ApplicationViewSet(ApplicationAttrsSerializerViewMixin, OrgBulkModelViewSet):
+class ApplicationViewSet(OrgBulkModelViewSet):
     model = models.Application
-    filter_fields = ('name', 'type', 'category')
-    search_fields = filter_fields
+    filterset_fields = ('name', 'type', 'category')
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.ApplicationSerializer
+

apps/applications/api/database_app.py

@@ -1,20 +0,0 @@
-# coding: utf-8
-# 
-
-from orgs.mixins.api import OrgBulkModelViewSet
-
-from .. import models
-from .. import serializers
-from ..hands import IsOrgAdminOrAppUser
-
-__all__ = [
-    'DatabaseAppViewSet',
-]
-
-
-class DatabaseAppViewSet(OrgBulkModelViewSet):
-    model = models.DatabaseApp
-    filter_fields = ('name',)
-    search_fields = filter_fields
-    permission_classes = (IsOrgAdminOrAppUser,)
-    serializer_class = serializers.DatabaseAppSerializer

apps/applications/api/k8s_app.py

@@ -1,20 +0,0 @@
-# coding: utf-8
-#
-
-from orgs.mixins.api import OrgBulkModelViewSet
-
-from .. import models
-from .. import serializers
-from ..hands import IsOrgAdminOrAppUser
-
-__all__ = [
-    'K8sAppViewSet',
-]
-
-
-class K8sAppViewSet(OrgBulkModelViewSet):
-    model = models.K8sApp
-    filter_fields = ('name',)
-    search_fields = filter_fields
-    permission_classes = (IsOrgAdminOrAppUser,)
-    serializer_class = serializers.K8sAppSerializer

apps/applications/api/mixin.py

@@ -1,57 +1,7 @@
-import uuid
-
-from common.exceptions import JMSException
 from orgs.models import Organization
-from .. import models
 
 
-class ApplicationAttrsSerializerViewMixin:
-
-    def get_serializer_class(self):
-        serializer_class = super().get_serializer_class()
-        if getattr(self, 'swagger_fake_view', False):
-            return serializer_class
-        app_type = self.request.query_params.get('type')
-        app_category = self.request.query_params.get('category')
-        type_options = list(dict(models.Category.get_all_type_serializer_mapper()).keys())
-        category_options = list(dict(models.Category.get_category_serializer_mapper()).keys())
-
-        # ListAPIView 没有 action 属性
-        # 不使用method属性，因为options请求时为method为post
-        action = getattr(self, 'action', 'list')
-
-        if app_type and app_type not in type_options:
-            raise JMSException(
-                'Invalid query parameter `type`, select from the following options: {}'
-                ''.format(type_options)
-            )
-        if app_category and app_category not in category_options:
-            raise JMSException(
-                'Invalid query parameter `category`, select from the following options: {}'
-                ''.format(category_options)
-            )
-
-        if action in [
-            'create', 'update', 'partial_update', 'bulk_update', 'partial_bulk_update'
-        ] and not app_type:
-            # action: create / update
-            raise JMSException(
-                'The `{}` action must take the `type` query parameter'.format(action)
-            )
-
-        if app_type:
-            # action: create / update / list / retrieve / metadata
-            attrs_cls = models.Category.get_type_serializer_cls(app_type)
-            class_name = 'ApplicationDynamicSerializer{}'.format(app_type.title())
-        elif app_category:
-            # action: list / retrieve / metadata
-            attrs_cls = models.Category.get_category_serializer_cls(app_category)
-            class_name = 'ApplicationDynamicSerializer{}'.format(app_category.title())
-        else:
-            attrs_cls = models.Category.get_no_password_serializer_cls()
-            class_name = 'ApplicationDynamicSerializer'
-        cls = type(class_name, (serializer_class,), {'attrs': attrs_cls()})
-        return cls
+__all__ = ['SerializeApplicationToTreeNodeMixin']
 
 
 class SerializeApplicationToTreeNodeMixin:

apps/applications/api/remote_app.py

@@ -1,40 +1,19 @@
 # coding: utf-8
 #
 
-from orgs.mixins.api import OrgBulkModelViewSet
 from orgs.mixins import generics
-from common.exceptions import JMSException
-from ..hands import IsOrgAdmin, IsAppUser
+from ..hands import IsAppUser
 from .. import models
-from ..serializers import RemoteAppSerializer, RemoteAppConnectionInfoSerializer
+from ..serializers import RemoteAppConnectionInfoSerializer
+from ..permissions import IsRemoteApp
 
 
 __all__ = [
-    'RemoteAppViewSet', 'RemoteAppConnectionInfoApi',
+    'RemoteAppConnectionInfoApi',
 ]
 
 
-class RemoteAppViewSet(OrgBulkModelViewSet):
-    model = models.RemoteApp
-    filter_fields = ('name', 'type', 'comment')
-    search_fields = filter_fields
-    permission_classes = (IsOrgAdmin,)
-    serializer_class = RemoteAppSerializer
-
-
 class RemoteAppConnectionInfoApi(generics.RetrieveAPIView):
     model = models.Application
-    permission_classes = (IsAppUser, )
+    permission_classes = (IsAppUser, IsRemoteApp)
     serializer_class = RemoteAppConnectionInfoSerializer
-
-    @staticmethod
-    def check_category_allowed(obj):
-        if not obj.category_is_remote_app:
-            raise JMSException(
-                'The request instance(`{}`) is not of category `remote_app`'.format(obj.category)
-            )
-
-    def get_object(self):
-        obj = super().get_object()
-        self.check_category_allowed(obj)
-        return obj

apps/applications/const.py

@@ -1,64 +1,49 @@
 #  coding: utf-8
 #
 
+from django.db.models import TextChoices
 from django.utils.translation import ugettext_lazy as _
 
 
-# RemoteApp
+class ApplicationCategoryChoices(TextChoices):
+    db = 'db', _('Database')
+    remote_app = 'remote_app', _('Remote app')
+    cloud = 'cloud', 'Cloud'
 
-REMOTE_APP_BOOT_PROGRAM_NAME = '||jmservisor'
-
-REMOTE_APP_TYPE_CHROME = 'chrome'
-REMOTE_APP_TYPE_MYSQL_WORKBENCH = 'mysql_workbench'
-REMOTE_APP_TYPE_VMWARE_CLIENT = 'vmware_client'
-REMOTE_APP_TYPE_CUSTOM = 'custom'
-
-# Fields attribute write_only default => False
-
-REMOTE_APP_TYPE_CHROME_FIELDS = [
-    {'name': 'chrome_target'},
-    {'name': 'chrome_username'},
-    {'name': 'chrome_password', 'write_only': True}
-]
-REMOTE_APP_TYPE_MYSQL_WORKBENCH_FIELDS = [
-    {'name': 'mysql_workbench_ip'},
-    {'name': 'mysql_workbench_name'},
-    {'name': 'mysql_workbench_port'},
-    {'name': 'mysql_workbench_username'},
-    {'name': 'mysql_workbench_password', 'write_only': True}
-]
-REMOTE_APP_TYPE_VMWARE_CLIENT_FIELDS = [
-    {'name': 'vmware_target'},
-    {'name': 'vmware_username'},
-    {'name': 'vmware_password', 'write_only': True}
-]
-REMOTE_APP_TYPE_CUSTOM_FIELDS = [
-    {'name': 'custom_cmdline'},
-    {'name': 'custom_target'},
-    {'name': 'custom_username'},
-    {'name': 'custom_password', 'write_only': True}
-]
-
-REMOTE_APP_TYPE_FIELDS_MAP = {
-    REMOTE_APP_TYPE_CHROME: REMOTE_APP_TYPE_CHROME_FIELDS,
-    REMOTE_APP_TYPE_MYSQL_WORKBENCH: REMOTE_APP_TYPE_MYSQL_WORKBENCH_FIELDS,
-    REMOTE_APP_TYPE_VMWARE_CLIENT: REMOTE_APP_TYPE_VMWARE_CLIENT_FIELDS,
-    REMOTE_APP_TYPE_CUSTOM: REMOTE_APP_TYPE_CUSTOM_FIELDS
-}
-
-REMOTE_APP_TYPE_CHOICES = (
-    (REMOTE_APP_TYPE_CHROME, 'Chrome'),
-    (REMOTE_APP_TYPE_MYSQL_WORKBENCH, 'MySQL Workbench'),
-    (REMOTE_APP_TYPE_VMWARE_CLIENT, 'vSphere Client'),
-    (REMOTE_APP_TYPE_CUSTOM, _('Custom')),
-)
+    @classmethod
+    def get_label(cls, category):
+        return dict(cls.choices).get(category, '')
 
 
-# DatabaseApp
+class ApplicationTypeChoices(TextChoices):
+    # db category
+    mysql = 'mysql', 'MySQL'
+    oracle = 'oracle', 'Oracle'
+    pgsql = 'postgresql', 'PostgreSQL'
+    mariadb = 'mariadb', 'MariaDB'
 
+    # remote-app category
+    chrome = 'chrome', 'Chrome'
+    mysql_workbench = 'mysql_workbench', 'MySQL Workbench'
+    vmware_client = 'vmware_client', 'vSphere Client'
+    custom = 'custom', _('Custom')
 
-DATABASE_APP_TYPE_MYSQL = 'mysql'
+    # cloud category
+    k8s = 'k8s', 'Kubernetes'
+
+    @classmethod
+    def get_label(cls, tp):
+        return dict(cls.choices).get(tp, '')
+
+    @classmethod
+    def db_types(cls):
+        return [cls.mysql.value, cls.oracle.value, cls.pgsql.value, cls.mariadb.value]
+
+    @classmethod
+    def remote_app_types(cls):
+        return [cls.chrome.value, cls.mysql_workbench.value, cls.vmware_client.value, cls.custom.value]
+
+    @classmethod
+    def cloud_types(cls):
+        return [cls.k8s.value]
 
-DATABASE_APP_TYPE_CHOICES = (
-    (DATABASE_APP_TYPE_MYSQL, 'MySQL'),
-)

apps/applications/migrations/0008_auto_20210104_0435.py

@@ -0,0 +1,28 @@
+# Generated by Django 3.1 on 2021-01-03 20:35
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('perms', '0017_auto_20210104_0435'),
+        ('applications', '0007_auto_20201119_1110'),
+    ]
+
+    operations = [
+        migrations.DeleteModel(
+            name='DatabaseApp',
+        ),
+        migrations.DeleteModel(
+            name='K8sApp',
+        ),
+        migrations.AlterField(
+            model_name='application',
+            name='attrs',
+            field=models.JSONField(default=dict, verbose_name='Attrs'),
+        ),
+        migrations.DeleteModel(
+            name='RemoteApp',
+        ),
+    ]

apps/applications/models/__init__.py

@@ -1,4 +1 @@
 from .application import *
-from .remote_app import *
-from .database_app import *
-from .k8s_app import *

apps/applications/models/application.py

@@ -1,128 +1,24 @@
-from itertools import chain
-
 from django.db import models
 from django.utils.translation import ugettext_lazy as _
 
 from orgs.mixins.models import OrgModelMixin
 from common.mixins import CommonModelMixin
-from common.db.models import ChoiceSet
-
-
-class DBType(ChoiceSet):
-    mysql = 'mysql', 'MySQL'
-    oracle = 'oracle', 'Oracle'
-    pgsql = 'postgresql', 'PostgreSQL'
-    mariadb = 'mariadb', 'MariaDB'
-
-    @classmethod
-    def get_type_serializer_cls_mapper(cls):
-        from ..serializers import database_app
-        mapper = {
-            cls.mysql: database_app.MySQLAttrsSerializer,
-            cls.oracle: database_app.OracleAttrsSerializer,
-            cls.pgsql: database_app.PostgreAttrsSerializer,
-            cls.mariadb: database_app.MariaDBAttrsSerializer,
-        }
-        return mapper
-
-
-class RemoteAppType(ChoiceSet):
-    chrome = 'chrome', 'Chrome'
-    mysql_workbench = 'mysql_workbench', 'MySQL Workbench'
-    vmware_client = 'vmware_client',  'vSphere Client'
-    custom = 'custom', _('Custom')
-
-    @classmethod
-    def get_type_serializer_cls_mapper(cls):
-        from ..serializers import remote_app
-        mapper = {
-            cls.chrome: remote_app.ChromeAttrsSerializer,
-            cls.mysql_workbench: remote_app.MySQLWorkbenchAttrsSerializer,
-            cls.vmware_client: remote_app.VMwareClientAttrsSerializer,
-            cls.custom: remote_app.CustomRemoteAppAttrsSeralizers,
-        }
-        return mapper
-
-
-class CloudType(ChoiceSet):
-    k8s = 'k8s', 'Kubernetes'
-
-    @classmethod
-    def get_type_serializer_cls_mapper(cls):
-        from ..serializers import k8s_app
-        mapper = {
-            cls.k8s: k8s_app.K8sAttrsSerializer,
-        }
-        return mapper
-
-
-class Category(ChoiceSet):
-    db = 'db', _('Database')
-    remote_app = 'remote_app', _('Remote app')
-    cloud = 'cloud', 'Cloud'
-
-    @classmethod
-    def get_category_type_mapper(cls):
-        return {
-            cls.db: DBType,
-            cls.remote_app: RemoteAppType,
-            cls.cloud: CloudType
-        }
-
-    @classmethod
-    def get_category_type_choices_mapper(cls):
-        return {
-            name: tp.choices
-            for name, tp in cls.get_category_type_mapper().items()
-        }
-
-    @classmethod
-    def get_type_choices(cls, category):
-        return cls.get_category_type_choices_mapper().get(category, [])
-
-    @classmethod
-    def get_all_type_choices(cls):
-        all_grouped_choices = tuple(cls.get_category_type_choices_mapper().values())
-        return tuple(chain(*all_grouped_choices))
-
-    @classmethod
-    def get_all_type_serializer_mapper(cls):
-        mapper = {}
-        for tp in cls.get_category_type_mapper().values():
-            mapper.update(tp.get_type_serializer_cls_mapper())
-        return mapper
-
-    @classmethod
-    def get_type_serializer_cls(cls, tp):
-        mapper = cls.get_all_type_serializer_mapper()
-        return mapper.get(tp, None)
-
-    @classmethod
-    def get_category_serializer_mapper(cls):
-        from ..serializers import remote_app, database_app, k8s_app
-        return {
-            cls.db: database_app.DBAttrsSerializer,
-            cls.remote_app: remote_app.RemoteAppAttrsSerializer,
-            cls.cloud: k8s_app.CloudAttrsSerializer,
-        }
-
-    @classmethod
-    def get_category_serializer_cls(cls, cg):
-        mapper = cls.get_category_serializer_mapper()
-        return mapper.get(cg, None)
-
-    @classmethod
-    def get_no_password_serializer_cls(cls):
-        from ..serializers import common
-        return common.NoPasswordSerializer
+from .. import const
 
 
 class Application(CommonModelMixin, OrgModelMixin):
     name = models.CharField(max_length=128, verbose_name=_('Name'))
-    domain = models.ForeignKey('assets.Domain', null=True, blank=True, related_name='applications', verbose_name=_("Domain"), on_delete=models.SET_NULL)
-    category = models.CharField(max_length=16, choices=Category.choices, verbose_name=_('Category'))
-    type = models.CharField(max_length=16, choices=Category.get_all_type_choices(), verbose_name=_('Type'))
-    attrs = models.JSONField()
+    category = models.CharField(
+        max_length=16, choices=const.ApplicationCategoryChoices.choices, verbose_name=_('Category')
+    )
+    type = models.CharField(
+        max_length=16, choices=const.ApplicationTypeChoices.choices, verbose_name=_('Type')
+    )
+    domain = models.ForeignKey(
+        'assets.Domain', null=True, blank=True, related_name='applications',
+        on_delete=models.SET_NULL, verbose_name=_("Domain"),
+    )
+    attrs = models.JSONField(default=dict, verbose_name=_('Attrs'))
     comment = models.TextField(
         max_length=128, default='', blank=True, verbose_name=_('Comment')
     )
@@ -136,5 +32,6 @@ class Application(CommonModelMixin, OrgModelMixin):
         type_display = self.get_type_display()
         return f'{self.name}({type_display})[{category_display}]'
 
-    def category_is_remote_app(self):
-        return self.category == Category.remote_app
+    @property
+    def category_remote_app(self):
+        return self.category == const.ApplicationCategoryChoices.remote_app.value

apps/applications/models/database_app.py

@@ -1,42 +0,0 @@
-# coding: utf-8
-#
-
-import uuid
-from django.db import models
-from django.utils.translation import ugettext_lazy as _
-
-from orgs.mixins.models import OrgModelMixin
-from common.mixins import CommonModelMixin
-from .. import const
-
-
-__all__ = ['DatabaseApp']
-
-
-class DatabaseApp(CommonModelMixin, OrgModelMixin):
-    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
-    name = models.CharField(max_length=128, verbose_name=_('Name'))
-    type = models.CharField(
-        default=const.DATABASE_APP_TYPE_MYSQL,
-        choices=const.DATABASE_APP_TYPE_CHOICES,
-        max_length=128, verbose_name=_('Type')
-    )
-    host = models.CharField(
-        max_length=128, verbose_name=_('Host'), db_index=True
-    )
-    port = models.IntegerField(default=3306, verbose_name=_('Port'))
-    database = models.CharField(
-        max_length=128, blank=True, null=True, verbose_name=_('Database'),
-        db_index=True
-    )
-    comment = models.TextField(
-        max_length=128, default='', blank=True, verbose_name=_('Comment')
-    )
-
-    def __str__(self):
-        return self.name
-
-    class Meta:
-        unique_together = [('org_id', 'name'), ]
-        verbose_name = _("DatabaseApp")
-        ordering = ('name', )

apps/applications/models/k8s_app.py

@@ -1,27 +0,0 @@
-from django.utils.translation import gettext_lazy as _
-
-from common.db import models
-from orgs.mixins.models import OrgModelMixin
-
-
-class K8sApp(OrgModelMixin, models.JMSModel):
-    class TYPE(models.ChoiceSet):
-        K8S = 'k8s', _('Kubernetes')
-
-    name = models.CharField(max_length=128, verbose_name=_('Name'))
-    type = models.CharField(
-        default=TYPE.K8S, choices=TYPE.choices,
-        max_length=128, verbose_name=_('Type')
-    )
-    cluster = models.CharField(max_length=1024, verbose_name=_('Cluster'))
-    comment = models.TextField(
-        max_length=128, default='', blank=True, verbose_name=_('Comment')
-    )
-
-    def __str__(self):
-        return self.name
-
-    class Meta:
-        unique_together = [('org_id', 'name'), ]
-        verbose_name = _('KubernetesApp')
-        ordering = ('name', )

apps/applications/models/remote_app.py

@@ -1,78 +0,0 @@
-# coding: utf-8
-#
-
-import uuid
-from django.db import models
-from django.utils.translation import ugettext_lazy as _
-
-from orgs.mixins.models import OrgModelMixin
-from common.fields.model import EncryptJsonDictTextField
-
-from .. import const
-
-
-__all__ = [
-    'RemoteApp',
-]
-
-
-class RemoteApp(OrgModelMixin):
-    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
-    name = models.CharField(max_length=128, verbose_name=_('Name'))
-    asset = models.ForeignKey(
-        'assets.Asset', on_delete=models.CASCADE, verbose_name=_('Asset')
-    )
-    type = models.CharField(
-        default=const.REMOTE_APP_TYPE_CHROME,
-        choices=const.REMOTE_APP_TYPE_CHOICES,
-        max_length=128, verbose_name=_('App type')
-    )
-    path = models.CharField(
-        max_length=128, blank=False, null=False,
-        verbose_name=_('App path')
-    )
-    params = EncryptJsonDictTextField(
-        max_length=4096, default={}, blank=True, null=True,
-        verbose_name=_('Parameters')
-    )
-    created_by = models.CharField(
-        max_length=32, null=True, blank=True, verbose_name=_('Created by')
-    )
-    date_created = models.DateTimeField(
-        auto_now_add=True, null=True, blank=True, verbose_name=_('Date created')
-    )
-    comment = models.TextField(
-        max_length=128, default='', blank=True, verbose_name=_('Comment')
-    )
-
-    class Meta:
-        verbose_name = _("RemoteApp")
-        unique_together = [('org_id', 'name')]
-        ordering = ('name', )
-
-    def __str__(self):
-        return self.name
-
-    @property
-    def parameters(self):
-        """
-        返回Guacamole需要的RemoteApp配置参数信息中的parameters参数
-        """
-        _parameters = list()
-        _parameters.append(self.type)
-        path = '\"%s\"' % self.path
-        _parameters.append(path)
-        for field in const.REMOTE_APP_TYPE_FIELDS_MAP[self.type]:
-            value = self.params.get(field['name'])
-            if value is None:
-                continue
-            _parameters.append(value)
-        _parameters = ' '.join(_parameters)
-        return _parameters
-
-    @property
-    def asset_info(self):
-        return {
-            'id': self.asset.id,
-            'hostname': self.asset.hostname
-        }

apps/applications/permissions.py

@@ -0,0 +1,9 @@
+from rest_framework import permissions
+
+
+__all__ = ['IsRemoteApp']
+
+
+class IsRemoteApp(permissions.BasePermission):
+    def has_object_permission(self, request, view, obj):
+        return obj.category_remote_app

apps/applications/serializers/__init__.py

@@ -1,5 +1,2 @@
 from .application import *
 from .remote_app import *
-from .database_app import *
-from .k8s_app import *
-from .common import *

apps/applications/serializers/application.py

@@ -4,15 +4,46 @@
 from rest_framework import serializers
 from django.utils.translation import ugettext_lazy as _
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
+from common.drf.serializers import MethodSerializer
+from .attrs import category_serializer_classes_mapping, type_serializer_classes_mapping
 
 from .. import models
 
 __all__ = [
-    'ApplicationSerializer',
+    'ApplicationSerializer', 'ApplicationSerializerMixin',
 ]
 
 
-class ApplicationSerializer(BulkOrgResourceModelSerializer):
+class ApplicationSerializerMixin(serializers.Serializer):
+    attrs = MethodSerializer()
+
+    def get_attrs_serializer(self):
+        default_serializer = serializers.Serializer(read_only=True)
+        if isinstance(self.instance, models.Application):
+            _type = self.instance.type
+            _category = self.instance.category
+        else:
+            _type = self.context['request'].query_params.get('type')
+            _category = self.context['request'].query_params.get('category')
+
+        if _type:
+            serializer_class = type_serializer_classes_mapping.get(_type)
+        elif _category:
+            serializer_class = category_serializer_classes_mapping.get(_category)
+        else:
+            serializer_class = default_serializer
+
+        if not serializer_class:
+            serializer_class = default_serializer
+
+        if isinstance(serializer_class, type):
+            serializer = serializer_class()
+        else:
+            serializer = serializer_class
+        return serializer
+
+
+class ApplicationSerializer(ApplicationSerializerMixin, BulkOrgResourceModelSerializer):
     category_display = serializers.ReadOnlyField(source='get_category_display', label=_('Category'))
     type_display = serializers.ReadOnlyField(source='get_type_display', label=_('Type'))
 
@@ -26,17 +57,8 @@ class ApplicationSerializer(BulkOrgResourceModelSerializer):
             'created_by', 'date_created', 'date_updated', 'get_type_display',
         ]
 
-    def create(self, validated_data):
-        validated_data['attrs'] = validated_data.pop('attrs', {})
-        instance = super().create(validated_data)
-        return instance
-
-    def update(self, instance, validated_data):
-        new_attrs = validated_data.pop('attrs', {})
-        instance = super().update(instance, validated_data)
-        attrs = instance.attrs
-        attrs.update(new_attrs)
-        instance.attrs = attrs
-        instance.save()
-        return instance
+    def validate_attrs(self, attrs):
+        _attrs = self.instance.attrs if self.instance else {}
+        _attrs.update(attrs)
+        return _attrs
 

apps/applications/serializers/attrs/__init__.py

@@ -0,0 +1 @@
+from .attrs import *

apps/applications/serializers/attrs/application_category/__init__.py

@@ -0,0 +1,3 @@
+from .remote_app import *
+from .db import *
+from .cloud import *

apps/applications/serializers/attrs/application_category/cloud.py

@@ -0,0 +1,9 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+
+__all__ = ['CloudSerializer']
+
+
+class CloudSerializer(serializers.Serializer):
+    cluster = serializers.CharField(max_length=1024, label=_('Cluster'), allow_null=True)

apps/applications/serializers/attrs/application_category/db.py

@@ -0,0 +1,15 @@
+# coding: utf-8
+#
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+
+__all__ = ['DBSerializer']
+
+
+class DBSerializer(serializers.Serializer):
+    host = serializers.CharField(max_length=128, label=_('Host'), allow_null=True)
+    port = serializers.IntegerField(label=_('Port'), allow_null=True)
+    database = serializers.CharField(
+        max_length=128, required=True, allow_null=True, label=_('Database')
+    )

apps/applications/serializers/attrs/application_category/remote_app.py

@@ -0,0 +1,52 @@
+# coding: utf-8
+#
+
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+from django.core.exceptions import ObjectDoesNotExist
+
+from common.utils import get_logger, is_uuid
+from assets.models import Asset
+
+logger = get_logger(__file__)
+
+
+__all__ = ['RemoteAppSerializer']
+
+
+class CharPrimaryKeyRelatedField(serializers.PrimaryKeyRelatedField):
+
+    def to_internal_value(self, data):
+        instance = super().to_internal_value(data)
+        return str(instance.id)
+
+    def to_representation(self, value):
+        # value is instance.id
+        if self.pk_field is not None:
+            return self.pk_field.to_representation(value)
+        return value
+
+
+class RemoteAppSerializer(serializers.Serializer):
+    asset_info = serializers.SerializerMethodField()
+    asset = CharPrimaryKeyRelatedField(
+        queryset=Asset.objects, required=False, label=_("Asset"), allow_null=True
+    )
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), allow_null=True
+    )
+
+    @staticmethod
+    def get_asset_info(obj):
+        asset_id = obj.get('asset')
+        if not asset_id or is_uuid(asset_id):
+            return {}
+        try:
+            asset = Asset.objects.filter(id=str(asset_id)).values_list('id', 'hostname')
+        except ObjectDoesNotExist as e:
+            logger.error(e)
+            return {}
+        if not asset:
+            return {}
+        asset_info = {'id': str(asset[0]), 'hostname': asset[1]}
+        return asset_info

apps/applications/serializers/attrs/application_type/__init__.py

@@ -0,0 +1,12 @@
+
+from .mysql import *
+from .mariadb import *
+from .oracle import *
+from .pgsql import *
+
+from .chrome import *
+from .mysql_workbench import *
+from .vmware_client import *
+from .custom import *
+
+from .k8s import *

apps/applications/serializers/attrs/application_type/chrome.py

@@ -0,0 +1,26 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['ChromeSerializer']
+
+
+class ChromeSerializer(RemoteAppSerializer):
+    CHROME_PATH = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'
+
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), default=CHROME_PATH, allow_null=True,
+    )
+    chrome_target = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Target URL'), allow_null=True,
+    )
+    chrome_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'), allow_null=True,
+    )
+    chrome_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True
+    )
+

apps/applications/serializers/attrs/application_type/custom.py

@@ -0,0 +1,27 @@
+
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['CustomSerializer']
+
+
+class CustomSerializer(RemoteAppSerializer):
+    custom_cmdline = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Operating parameter'),
+        allow_null=True,
+    )
+    custom_target = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Target url'),
+        allow_null=True,
+    )
+    custom_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'),
+        allow_null=True,
+    )
+    custom_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True,
+    )

apps/applications/serializers/attrs/application_type/k8s.py

@@ -0,0 +1,8 @@
+from ..application_category import CloudSerializer
+
+
+__all__ = ['K8SSerializer']
+
+
+class K8SSerializer(CloudSerializer):
+    pass

apps/applications/serializers/attrs/application_type/mariadb.py

@@ -0,0 +1,8 @@
+from .mysql import MySQLSerializer
+
+
+__all__ = ['MariaDBSerializer']
+
+
+class MariaDBSerializer(MySQLSerializer):
+    pass

apps/applications/serializers/attrs/application_type/mysql.py

@@ -0,0 +1,15 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+from ..application_category import DBSerializer
+
+
+__all__ = ['MySQLSerializer']
+
+
+class MySQLSerializer(DBSerializer):
+    port = serializers.IntegerField(default=3306, label=_('Port'), allow_null=True)
+
+
+
+

apps/applications/serializers/attrs/application_type/mysql_workbench.py

@@ -0,0 +1,36 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['MySQLWorkbenchSerializer']
+
+
+class MySQLWorkbenchSerializer(RemoteAppSerializer):
+    MYSQL_WORKBENCH_PATH = 'C:\Program Files\MySQL\MySQL Workbench 8.0 CE\MySQLWorkbench.exe'
+
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), default=MYSQL_WORKBENCH_PATH,
+        allow_null=True,
+    )
+    mysql_workbench_ip = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('IP'),
+        allow_null=True,
+    )
+    mysql_workbench_port = serializers.IntegerField(
+        required=False, label=_('Port'),
+        allow_null=True,
+    )
+    mysql_workbench_name = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Database'),
+        allow_null=True,
+    )
+    mysql_workbench_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'),
+        allow_null=True,
+    )
+    mysql_workbench_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True,
+    )

apps/applications/serializers/attrs/application_type/oracle.py

@@ -0,0 +1,12 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+from ..application_category import DBSerializer
+
+
+__all__ = ['OracleSerializer']
+
+
+class OracleSerializer(DBSerializer):
+    port = serializers.IntegerField(default=1521, label=_('Port'), allow_null=True)
+

apps/applications/serializers/attrs/application_type/pgsql.py

@@ -0,0 +1,12 @@
+from rest_framework import serializers
+from django.utils.translation import ugettext_lazy as _
+
+from ..application_category import DBSerializer
+
+
+__all__ = ['PostgreSerializer']
+
+
+class PostgreSerializer(DBSerializer):
+    port = serializers.IntegerField(default=5432, label=_('Port'), allow_null=True)
+

apps/applications/serializers/attrs/application_type/vmware_client.py

@@ -0,0 +1,32 @@
+from django.utils.translation import ugettext_lazy as _
+from rest_framework import serializers
+
+from ..application_category import RemoteAppSerializer
+
+
+__all__ = ['VMwareClientSerializer']
+
+
+class VMwareClientSerializer(RemoteAppSerializer):
+    PATH = r'''
+    C:\Program Files (x86)\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\VpxClient
+    .exe
+    '''
+    VMWARE_CLIENT_PATH = ''.join(PATH.split())
+
+    path = serializers.CharField(
+        max_length=128, label=_('Application path'), default=VMWARE_CLIENT_PATH,
+        allow_null=True
+    )
+    vmware_target = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Target URL'),
+        allow_null=True
+    )
+    vmware_username = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, label=_('Username'),
+        allow_null=True
+    )
+    vmware_password = serializers.CharField(
+        max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'),
+        allow_null=True
+    )

apps/applications/serializers/attrs/attrs.py

@@ -0,0 +1,42 @@
+from rest_framework import serializers
+from applications import const
+from . import application_category, application_type
+
+
+__all__ = [
+    'category_serializer_classes_mapping',
+    'type_serializer_classes_mapping',
+    'get_serializer_class_by_application_type',
+]
+
+
+# define `attrs` field `category serializers mapping`
+# ---------------------------------------------------
+
+category_serializer_classes_mapping = {
+    const.ApplicationCategoryChoices.db.value: application_category.DBSerializer,
+    const.ApplicationCategoryChoices.remote_app.value: application_category.RemoteAppSerializer,
+    const.ApplicationCategoryChoices.cloud.value: application_category.CloudSerializer,
+}
+
+# define `attrs` field `type serializers mapping`
+# -----------------------------------------------
+
+type_serializer_classes_mapping = {
+    # db
+    const.ApplicationTypeChoices.mysql.value: application_type.MySQLSerializer,
+    const.ApplicationTypeChoices.mariadb.value: application_type.MariaDBSerializer,
+    const.ApplicationTypeChoices.oracle.value: application_type.OracleSerializer,
+    const.ApplicationTypeChoices.pgsql.value: application_type.PostgreSerializer,
+    # remote-app
+    const.ApplicationTypeChoices.chrome.value: application_type.ChromeSerializer,
+    const.ApplicationTypeChoices.mysql_workbench.value: application_type.MySQLWorkbenchSerializer,
+    const.ApplicationTypeChoices.vmware_client.value: application_type.VMwareClientSerializer,
+    const.ApplicationTypeChoices.custom.value: application_type.CustomSerializer,
+    # cloud
+    const.ApplicationTypeChoices.k8s.value: application_type.K8SSerializer
+}
+
+
+def get_serializer_class_by_application_type(_application_type):
+    return type_serializer_classes_mapping.get(_application_type)

apps/applications/serializers/common.py

@@ -1,11 +0,0 @@
-from rest_framework import serializers
-
-
-class NoPasswordSerializer(serializers.JSONField):
-    def to_representation(self, value):
-        new_value = {}
-        for k, v in value.items():
-            if 'password' not in k:
-                new_value[k] = v
-        return new_value
-

apps/applications/serializers/database_app.py

@@ -1,50 +0,0 @@
-# coding: utf-8
-#
-from rest_framework import serializers
-from django.utils.translation import ugettext_lazy as _
-
-from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-from common.serializers import AdaptedBulkListSerializer
-
-from .. import models
-
-
-class DBAttrsSerializer(serializers.Serializer):
-    host = serializers.CharField(max_length=128, label=_('Host'))
-    port = serializers.IntegerField(label=_('Port'))
-    # 添加allow_null=True，兼容之前数据库中database字段为None的情况
-    database = serializers.CharField(max_length=128, required=True, allow_null=True, label=_('Database'))
-
-
-class MySQLAttrsSerializer(DBAttrsSerializer):
-    port = serializers.IntegerField(default=3306, label=_('Port'))
-
-
-class PostgreAttrsSerializer(DBAttrsSerializer):
-    port = serializers.IntegerField(default=5432, label=_('Port'))
-
-
-class OracleAttrsSerializer(DBAttrsSerializer):
-    port = serializers.IntegerField(default=1521, label=_('Port'))
-
-
-class MariaDBAttrsSerializer(MySQLAttrsSerializer):
-    pass
-
-
-class DatabaseAppSerializer(BulkOrgResourceModelSerializer):
-
-    class Meta:
-        model = models.DatabaseApp
-        list_serializer_class = AdaptedBulkListSerializer
-        fields = [
-            'id', 'name', 'type', 'get_type_display', 'host', 'port',
-            'database', 'comment', 'created_by', 'date_created', 'date_updated',
-        ]
-        read_only_fields = [
-            'created_by', 'date_created', 'date_updated'
-            'get_type_display',
-        ]
-        extra_kwargs = {
-            'get_type_display': {'label': _('Type for display')},
-        }

apps/applications/serializers/k8s_app.py

@@ -1,27 +0,0 @@
-from rest_framework import serializers
-from django.utils.translation import ugettext_lazy as _
-
-from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-from .. import models
-
-
-class CloudAttrsSerializer(serializers.Serializer):
-    cluster = serializers.CharField(max_length=1024, label=_('Cluster'))
-
-
-class K8sAttrsSerializer(CloudAttrsSerializer):
-    pass
-
-
-class K8sAppSerializer(BulkOrgResourceModelSerializer):
-    type_display = serializers.CharField(source='get_type_display', read_only=True, label=_('Type for display'))
-
-    class Meta:
-        model = models.K8sApp
-        fields = [
-            'id', 'name', 'type', 'type_display', 'comment', 'created_by',
-            'date_created', 'date_updated', 'cluster'
-        ]
-        read_only_fields = [
-            'id', 'created_by', 'date_created', 'date_updated',
-        ]

apps/applications/serializers/remote_app.py

@@ -1,89 +1,14 @@
 # coding: utf-8
 #
-
-import copy
-from django.utils.translation import ugettext_lazy as _
-from django.core.exceptions import ObjectDoesNotExist
 from rest_framework import serializers
-
-from common.serializers import AdaptedBulkListSerializer
-from common.fields.serializer import CustomMetaDictField
 from common.utils import get_logger
-from orgs.mixins.serializers import BulkOrgResourceModelSerializer
-from assets.models import Asset
+from ..models import Application
 
-from .. import const
-from ..models import RemoteApp, Category, Application
 
 logger = get_logger(__file__)
 
 
-class CharPrimaryKeyRelatedField(serializers.PrimaryKeyRelatedField):
-
-    def to_internal_value(self, data):
-        instance = super().to_internal_value(data)
-        return str(instance.id)
-
-    def to_representation(self, value):
-        # value is instance.id
-        if self.pk_field is not None:
-            return self.pk_field.to_representation(value)
-        return value
-
-
-class RemoteAppAttrsSerializer(serializers.Serializer):
-    asset_info = serializers.SerializerMethodField()
-    asset = CharPrimaryKeyRelatedField(queryset=Asset.objects, required=False, label=_("Asset"))
-    path = serializers.CharField(max_length=128, label=_('Application path'))
-
-    @staticmethod
-    def get_asset_info(obj):
-        asset_info = {}
-        asset_id = obj.get('asset')
-        if not asset_id:
-            return asset_info
-        try:
-            asset = Asset.objects.get(id=asset_id)
-            asset_info.update({
-                'id': str(asset.id),
-                'hostname': asset.hostname
-            })
-        except ObjectDoesNotExist as e:
-            logger.error(e)
-        return asset_info
-
-
-class ChromeAttrsSerializer(RemoteAppAttrsSerializer):
-    REMOTE_APP_PATH = 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'
-    path = serializers.CharField(max_length=128, label=_('Application path'), default=REMOTE_APP_PATH)
-    chrome_target = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('Target URL'))
-    chrome_username = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('Username'))
-    chrome_password = serializers.CharField(max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'))
-
-
-class MySQLWorkbenchAttrsSerializer(RemoteAppAttrsSerializer):
-    REMOTE_APP_PATH = 'C:\Program Files\MySQL\MySQL Workbench 8.0 CE\MySQLWorkbench.exe'
-    path = serializers.CharField(max_length=128, label=_('Application path'), default=REMOTE_APP_PATH)
-    mysql_workbench_ip = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('IP'))
-    mysql_workbench_port = serializers.IntegerField(required=False, label=_('Port'))
-    mysql_workbench_name = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('Database'))
-    mysql_workbench_username = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('Username'))
-    mysql_workbench_password = serializers.CharField(max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'))
-
-
-class VMwareClientAttrsSerializer(RemoteAppAttrsSerializer):
-    REMOTE_APP_PATH = 'C:\Program Files (x86)\VMware\Infrastructure\Virtual Infrastructure Client\Launcher\VpxClient.exe'
-    path = serializers.CharField(max_length=128, label=_('Application path'), default=REMOTE_APP_PATH)
-    vmware_target = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('Target URL'))
-    vmware_username = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('Username'))
-    vmware_password = serializers.CharField(max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'))
-
-
-class CustomRemoteAppAttrsSeralizers(RemoteAppAttrsSerializer):
-    custom_cmdline = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('Operating parameter'))
-    custom_target = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('Target url'))
-    custom_username = serializers.CharField(max_length=128, allow_blank=True, required=False, label=_('Username'))
-    custom_password = serializers.CharField(max_length=128, allow_blank=True, required=False, write_only=True, label=_('Password'))
+__all__ = ['RemoteAppConnectionInfoSerializer']
 
 
 class RemoteAppConnectionInfoSerializer(serializers.ModelSerializer):
@@ -97,94 +22,36 @@ class RemoteAppConnectionInfoSerializer(serializers.ModelSerializer):
         ]
         read_only_fields = ['parameter_remote_app']
 
+    @staticmethod
+    def get_asset(obj):
+        return obj.attrs.get('asset')
+
     @staticmethod
     def get_parameters(obj):
         """
         返回Guacamole需要的RemoteApp配置参数信息中的parameters参数
         """
-        serializer_cls = Category.get_type_serializer_cls(obj.type)
-        fields = serializer_cls().get_fields()
-        fields.pop('asset', None)
-        fields_name = list(fields.keys())
-        attrs = obj.attrs
-        _parameters = list()
-        _parameters.append(obj.type)
-        for field_name in list(fields_name):
-            value = attrs.get(field_name, None)
+        from .attrs import get_serializer_class_by_application_type
+        serializer_class = get_serializer_class_by_application_type(obj.type)
+        fields = serializer_class().get_fields()
+
+        parameters = [obj.type]
+        for field_name in list(fields.keys()):
+            if field_name in ['asset']:
+                continue
+            value = obj.attrs.get(field_name)
             if not value:
                 continue
             if field_name == 'path':
                 value = '\"%s\"' % value
-            _parameters.append(str(value))
-        _parameters = ' '.join(_parameters)
-        return _parameters
+            parameters.append(str(value))
+
+        parameters = ' '.join(parameters)
+        return parameters
 
     def get_parameter_remote_app(self, obj):
-        parameters = self.get_parameters(obj)
-        parameter = {
-            'program': const.REMOTE_APP_BOOT_PROGRAM_NAME,
+        return {
+            'program': '||jmservisor',
             'working_directory': '',
-            'parameters': parameters,
+            'parameters': self.get_parameters(obj)
         }
-        return parameter
-
-    @staticmethod
-    def get_asset(obj):
-        return obj.attrs.get('asset')
-
-
-# TODO: DELETE
-class RemoteAppParamsDictField(CustomMetaDictField):
-    type_fields_map = const.REMOTE_APP_TYPE_FIELDS_MAP
-    default_type = const.REMOTE_APP_TYPE_CHROME
-    convert_key_remove_type_prefix = False
-    convert_key_to_upper = False
-
-
-# TODO: DELETE
-class RemoteAppSerializer(BulkOrgResourceModelSerializer):
-    params = RemoteAppParamsDictField(label=_('Parameters'))
-    type_fields_map = const.REMOTE_APP_TYPE_FIELDS_MAP
-
-    class Meta:
-        model = RemoteApp
-        list_serializer_class = AdaptedBulkListSerializer
-        fields = [
-            'id', 'name', 'asset', 'asset_info', 'type', 'get_type_display',
-            'path', 'params', 'date_created', 'created_by', 'comment',
-        ]
-        read_only_fields = [
-            'created_by', 'date_created', 'asset_info',
-            'get_type_display'
-        ]
-        extra_kwargs = {
-            'asset_info': {'label': _('Asset info')},
-            'get_type_display': {'label': _('Type for display')},
-        }
-
-    def process_params(self, instance, validated_data):
-        new_params = copy.deepcopy(validated_data.get('params', {}))
-        tp = validated_data.get('type', '')
-
-        if tp != instance.type:
-            return new_params
-
-        old_params = instance.params
-        fields = self.type_fields_map.get(instance.type, [])
-        for field in fields:
-            if not field.get('write_only', False):
-                continue
-            field_name = field['name']
-            new_value = new_params.get(field_name, '')
-            old_value = old_params.get(field_name, '')
-            field_value = new_value if new_value else old_value
-            new_params[field_name] = field_value
-
-        return new_params
-
-    def update(self, instance, validated_data):
-        params = self.process_params(instance, validated_data)
-        validated_data['params'] = params
-        return super().update(instance, validated_data)
-
-

apps/applications/urls/api_urls.py

@@ -1,26 +1,20 @@
 # coding:utf-8
 #
-
-from django.urls import path, re_path
+from django.urls import path
 from rest_framework_bulk.routes import BulkRouter
-
-from common import api as capi
 from .. import api
 
+
 app_name = 'applications'
 
+
 router = BulkRouter()
 router.register(r'applications', api.ApplicationViewSet, 'application')
-router.register(r'remote-apps', api.RemoteAppViewSet, 'remote-app')
-router.register(r'database-apps', api.DatabaseAppViewSet, 'database-app')
-router.register(r'k8s-apps', api.K8sAppViewSet, 'k8s-app')
+
 
 urlpatterns = [
     path('remote-apps/<uuid:pk>/connection-info/', api.RemoteAppConnectionInfoApi.as_view(), name='remote-app-connection-info'),
 ]
 
-old_version_urlpatterns = [
-    re_path('(?P<resource>remote-app)/.*', capi.redirect_plural_name_api)
-]
 
-urlpatterns += router.urls + old_version_urlpatterns
+urlpatterns += router.urls

apps/applications/urls/views_urls.py

@@ -1,7 +0,0 @@
-# coding:utf-8
-from django.urls import path
-
-app_name = 'applications'
-
-urlpatterns = [
-]

apps/assets/api/admin_user.py

@@ -29,8 +29,8 @@ class AdminUserViewSet(OrgBulkModelViewSet):
     Admin user api set, for add,delete,update,list,retrieve resource
     """
     model = AdminUser
-    filter_fields = ("name", "username")
-    search_fields = filter_fields
+    filterset_fields = ("name", "username")
+    search_fields = filterset_fields
     serializer_class = serializers.AdminUserSerializer
     permission_classes = (IsOrgAdmin,)
 
@@ -93,8 +93,8 @@ class AdminUserTestConnectiveApi(generics.RetrieveAPIView):
 class AdminUserAssetsListView(generics.ListAPIView):
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.AssetSimpleSerializer
-    filter_fields = ("hostname", "ip")
-    search_fields = filter_fields
+    filterset_fields = ("hostname", "ip")
+    search_fields = filterset_fields
 
     def get_object(self):
         pk = self.kwargs.get('pk')

apps/assets/api/asset.py

@@ -3,6 +3,8 @@
 from assets.api import FilterAssetByNodeMixin
 from rest_framework.viewsets import ModelViewSet
 from rest_framework.generics import RetrieveAPIView
+from rest_framework.response import Response
+from rest_framework import status
 from django.shortcuts import get_object_or_404
 
 from common.utils import get_logger, get_object_or_none
@@ -12,7 +14,7 @@ from orgs.mixins import generics
 from ..models import Asset, Node, Platform
 from .. import serializers
 from ..tasks import (
-    update_asset_hardware_info_manual, test_asset_connectivity_manual
+    update_assets_hardware_info_manual, test_assets_connectivity_manual
 )
 from ..filters import FilterAssetByNodeFilterBackend, LabelFilterBackend, IpInFilterBackend
 
@@ -21,7 +23,7 @@ logger = get_logger(__file__)
 __all__ = [
     'AssetViewSet', 'AssetPlatformRetrieveApi',
     'AssetGatewayListApi', 'AssetPlatformViewSet',
-    'AssetTaskCreateApi',
+    'AssetTaskCreateApi', 'AssetsTaskCreateApi',
 ]
 
 
@@ -30,10 +32,15 @@ class AssetViewSet(FilterAssetByNodeMixin, OrgBulkModelViewSet):
     API endpoint that allows Asset to be viewed or edited.
     """
     model = Asset
-    filter_fields = (
-        "hostname", "ip", "systemuser__id", "admin_user__id", "platform__base",
-        "is_active"
-    )
+    filterset_fields = {
+        'hostname': ['exact'],
+        'ip': ['exact'],
+        'systemuser__id': ['exact'],
+        'admin_user__id': ['exact'],
+        'platform__base': ['exact'],
+        'is_active': ['exact'],
+        'protocols': ['exact', 'icontains']
+    }
     search_fields = ("hostname", "ip")
     ordering_fields = ("hostname", "ip", "port", "cpu_cores")
     serializer_classes = {
@@ -74,7 +81,7 @@ class AssetPlatformViewSet(ModelViewSet):
     queryset = Platform.objects.all()
     permission_classes = (IsSuperUser,)
     serializer_class = serializers.PlatformSerializer
-    filter_fields = ['name', 'base']
+    filterset_fields = ['name', 'base']
     search_fields = ['name']
 
     def get_permissions(self):
@@ -90,26 +97,38 @@ class AssetPlatformViewSet(ModelViewSet):
         return super().check_object_permissions(request, obj)
 
 
-class AssetTaskCreateApi(generics.CreateAPIView):
+class AssetsTaskMixin:
+    def perform_assets_task(self, serializer):
+        data = serializer.validated_data
+        assets = data['assets']
+        action = data['action']
+        if action == "refresh":
+            task = update_assets_hardware_info_manual.delay(assets)
+        else:
+            task = test_assets_connectivity_manual.delay(assets)
+        data = getattr(serializer, '_data', {})
+        data["task"] = task.id
+        setattr(serializer, '_data', data)
+
+    def perform_create(self, serializer):
+        self.perform_assets_task(serializer)
+
+
+class AssetTaskCreateApi(AssetsTaskMixin, generics.CreateAPIView):
     model = Asset
     serializer_class = serializers.AssetTaskSerializer
     permission_classes = (IsOrgAdmin,)
 
-    def get_object(self):
-        pk = self.kwargs.get("pk")
-        instance = get_object_or_404(Asset, pk=pk)
-        return instance
+    def create(self, request, *args, **kwargs):
+        pk = self.kwargs.get('pk')
+        request.data['assets'] = [pk]
+        return super().create(request, *args, **kwargs)
 
-    def perform_create(self, serializer):
-        asset = self.get_object()
-        action = serializer.validated_data["action"]
-        if action == "refresh":
-            task = update_asset_hardware_info_manual.delay(asset)
-        else:
-            task = test_asset_connectivity_manual.delay(asset)
-        data = getattr(serializer, '_data', {})
-        data["task"] = task.id
-        setattr(serializer, '_data', data)
+
+class AssetsTaskCreateApi(AssetsTaskMixin, generics.CreateAPIView):
+    model = Asset
+    serializer_class = serializers.AssetTaskSerializer
+    permission_classes = (IsOrgAdmin,)
 
 
 class AssetGatewayListApi(generics.ListAPIView):

apps/assets/api/asset_user.py

@@ -28,7 +28,7 @@ logger = get_logger(__name__)
 class AssetUserFilterBackend(filters.BaseFilterBackend):
     def filter_queryset(self, request, queryset, view):
         kwargs = {}
-        for field in view.filter_fields:
+        for field in view.filterset_fields:
             value = request.GET.get(field)
             if not value:
                 continue
@@ -78,7 +78,7 @@ class AssetUserViewSet(CommonApiMixin, BulkModelViewSet):
         'retrieve': serializers.AssetUserReadSerializer,
     }
     permission_classes = [IsOrgAdminOrAppUser]
-    filter_fields = [
+    filterset_fields = [
         "id", "ip", "hostname", "username",
         "asset_id", "node_id",
         "prefer", "prefer_id",
@@ -131,7 +131,7 @@ class AssetUserTaskCreateAPI(generics.CreateAPIView):
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.AssetUserTaskSerializer
     filter_backends = AssetUserViewSet.filter_backends
-    filter_fields = AssetUserViewSet.filter_fields
+    filterset_fields = AssetUserViewSet.filterset_fields
 
     def get_asset_users(self):
         manager = AssetUserManager()

apps/assets/api/cmd_filter.py

@@ -14,16 +14,16 @@ __all__ = ['CommandFilterViewSet', 'CommandFilterRuleViewSet']
 
 class CommandFilterViewSet(OrgBulkModelViewSet):
     model = CommandFilter
-    filter_fields = ("name",)
-    search_fields = filter_fields
+    filterset_fields = ("name",)
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.CommandFilterSerializer
 
 
 class CommandFilterRuleViewSet(OrgBulkModelViewSet):
     model = CommandFilterRule
-    filter_fields = ("content",)
-    search_fields = filter_fields
+    filterset_fields = ("content",)
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.CommandFilterRuleSerializer
 

apps/assets/api/domain.py

@@ -18,8 +18,8 @@ __all__ = ['DomainViewSet', 'GatewayViewSet', "GatewayTestConnectionApi"]
 
 class DomainViewSet(OrgBulkModelViewSet):
     model = Domain
-    filter_fields = ("name", )
-    search_fields = filter_fields
+    filterset_fields = ("name", )
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.DomainSerializer
 
@@ -31,7 +31,7 @@ class DomainViewSet(OrgBulkModelViewSet):
 
 class GatewayViewSet(OrgBulkModelViewSet):
     model = Gateway
-    filter_fields = ("domain__name", "name", "username", "ip", "domain")
+    filterset_fields = ("domain__name", "name", "username", "ip", "domain")
     search_fields = ("domain__name", "name", "username", "ip")
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.GatewaySerializer

apps/assets/api/favorite_asset.py

@@ -13,7 +13,7 @@ __all__ = ['FavoriteAssetViewSet']
 class FavoriteAssetViewSet(BulkModelViewSet):
     serializer_class = FavoriteAssetSerializer
     permission_classes = (IsValidUser,)
-    filter_fields = ['asset']
+    filterset_fields = ['asset']
 
     def dispatch(self, request, *args, **kwargs):
         with tmp_to_root_org():

apps/assets/api/gathered_user.py

@@ -18,5 +18,5 @@ class GatheredUserViewSet(OrgModelViewSet):
     permission_classes = [IsOrgAdmin]
     extra_filter_backends = [AssetRelatedByNodeFilterBackend]
 
-    filter_fields = ['asset', 'username', 'present', 'asset__ip', 'asset__hostname', 'asset_id']
+    filterset_fields = ['asset', 'username', 'present', 'asset__ip', 'asset__hostname', 'asset_id']
     search_fields = ['username', 'asset__ip', 'asset__hostname']

apps/assets/api/label.py

@@ -28,8 +28,8 @@ __all__ = ['LabelViewSet']
 
 class LabelViewSet(OrgBulkModelViewSet):
     model = Label
-    filter_fields = ("name", "value")
-    search_fields = filter_fields
+    filterset_fields = ("name", "value")
+    search_fields = filterset_fields
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.LabelSerializer
 

apps/assets/api/node.py

@@ -45,7 +45,7 @@ __all__ = [
 
 class NodeViewSet(OrgModelViewSet):
     model = Node
-    filter_fields = ('value', 'key', 'id')
+    filterset_fields = ('value', 'key', 'id')
     search_fields = ('value', )
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.NodeSerializer

apps/assets/api/system_user.py

@@ -29,8 +29,12 @@ class SystemUserViewSet(OrgBulkModelViewSet):
     System user api set, for add,delete,update,list,retrieve resource
     """
     model = SystemUser
-    filter_fields = ("name", "username", "protocol")
-    search_fields = filter_fields
+    filterset_fields = {
+        'name': ['exact'],
+        'username': ['exact'],
+        'protocol': ['exact', 'in']
+    }
+    search_fields = filterset_fields
     serializer_class = serializers.SystemUserSerializer
     serializer_classes = {
         'default': serializers.SystemUserSerializer,
@@ -136,8 +140,8 @@ class SystemUserCommandFilterRuleListApi(generics.ListAPIView):
 class SystemUserAssetsListView(generics.ListAPIView):
     permission_classes = (IsOrgAdmin,)
     serializer_class = serializers.AssetSimpleSerializer
-    filter_fields = ("hostname", "ip")
-    search_fields = filter_fields
+    filterset_fields = ("hostname", "ip")
+    search_fields = filterset_fields
 
     def get_object(self):
         pk = self.kwargs.get('pk')

apps/assets/api/system_user_relation.py

@@ -65,7 +65,7 @@ class SystemUserAssetRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserAssetRelationSerializer
     model = models.SystemUser.assets.through
     permission_classes = (IsOrgAdmin,)
-    filter_fields = [
+    filterset_fields = [
         'id', 'asset', 'systemuser',
     ]
     search_fields = [
@@ -91,7 +91,7 @@ class SystemUserNodeRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserNodeRelationSerializer
     model = models.SystemUser.nodes.through
     permission_classes = (IsOrgAdmin,)
-    filter_fields = [
+    filterset_fields = [
         'id', 'node', 'systemuser',
     ]
     search_fields = [
@@ -112,7 +112,7 @@ class SystemUserUserRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserUserRelationSerializer
     model = models.SystemUser.users.through
     permission_classes = (IsOrgAdmin,)
-    filter_fields = [
+    filterset_fields = [
         'id', 'user', 'systemuser',
     ]
     search_fields = [

apps/assets/migrations/0065_auto_20210121_1549.py

@@ -0,0 +1,17 @@
+# Generated by Django 3.1 on 2021-01-21 07:49
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0064_auto_20201203_1100'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='domain',
+            options={'ordering': ('name',), 'verbose_name': 'Domain'},
+        ),
+    ]

apps/assets/models/authbook.py

@@ -57,7 +57,7 @@ class AuthBook(BaseUser):
         同时设置自己的 is_latest=True, version=max_version + 1
         """
         username = kwargs['username']
-        asset = kwargs['asset']
+        asset = kwargs.get('asset') or kwargs.get('asset_id')
         with transaction.atomic():
             # 使用select_for_update限制并发创建相同的username、asset条目
             instances = cls.objects.select_for_update().filter(username=username, asset=asset)

apps/assets/models/domain.py

@@ -26,6 +26,7 @@ class Domain(OrgModelMixin):
     class Meta:
         verbose_name = _("Domain")
         unique_together = [('org_id', 'name')]
+        ordering = ('name',)
 
     def __str__(self):
         return self.name

apps/assets/models/node.py

@@ -38,6 +38,7 @@ class FamilyMixin:
     __children = None
     __all_children = None
     is_node = True
+    child_mark: int
 
     @staticmethod
     def clean_children_keys(nodes_keys):
@@ -121,11 +122,22 @@ class FamilyMixin:
             created = True
         return child, created
 
+    def get_valid_child_mark(self):
+        key = "{}:{}".format(self.key, self.child_mark)
+        if not self.__class__.objects.filter(key=key).exists():
+            return self.child_mark
+        children_keys = self.get_children().values_list('key', flat=True)
+        children_keys_last = [key.split(':')[-1] for key in children_keys]
+        children_keys_last = [int(k) for k in children_keys_last if k.strip().isdigit()]
+        max_key_last = max(children_keys_last) if children_keys_last else 1
+        return max_key_last + 1
+
     def get_next_child_key(self):
-        mark = self.child_mark
-        self.child_mark += 1
+        child_mark = self.get_valid_child_mark()
+        key = "{}:{}".format(self.key, child_mark)
+        self.child_mark = child_mark + 1
         self.save()
-        return "{}:{}".format(self.key, mark)
+        return key
 
     def get_next_child_preset_name(self):
         name = ugettext("New node")

apps/assets/models/user.py

@@ -87,6 +87,23 @@ class SystemUser(BaseUser):
         (PROTOCOL_POSTGRESQL, 'postgresql'),
         (PROTOCOL_K8S, 'k8s'),
     )
+    ASSET_CATEGORY_PROTOCOLS = [
+        PROTOCOL_SSH, PROTOCOL_RDP, PROTOCOL_TELNET, PROTOCOL_VNC
+    ]
+    APPLICATION_CATEGORY_REMOTE_APP_PROTOCOLS = [
+        PROTOCOL_RDP
+    ]
+    APPLICATION_CATEGORY_DB_PROTOCOLS = [
+        PROTOCOL_MYSQL, PROTOCOL_ORACLE, PROTOCOL_MARIADB, PROTOCOL_POSTGRESQL
+    ]
+    APPLICATION_CATEGORY_CLOUD_PROTOCOLS = [
+        PROTOCOL_K8S
+    ]
+    APPLICATION_CATEGORY_PROTOCOLS = [
+        *APPLICATION_CATEGORY_REMOTE_APP_PROTOCOLS,
+        *APPLICATION_CATEGORY_DB_PROTOCOLS,
+        *APPLICATION_CATEGORY_CLOUD_PROTOCOLS
+    ]
 
     LOGIN_AUTO = 'auto'
     LOGIN_MANUAL = 'manual'
@@ -133,24 +150,6 @@ class SystemUser(BaseUser):
     def login_mode_display(self):
         return self.get_login_mode_display()
 
-    @property
-    def db_application_protocols(self):
-        return [
-            self.PROTOCOL_MYSQL, self.PROTOCOL_ORACLE, self.PROTOCOL_MARIADB,
-            self.PROTOCOL_POSTGRESQL
-        ]
-
-    @property
-    def cloud_application_protocols(self):
-        return [self.PROTOCOL_K8S]
-
-    @property
-    def application_category_protocols(self):
-        protocols = []
-        protocols.extend(self.db_application_protocols)
-        protocols.extend(self.cloud_application_protocols)
-        return protocols
-
     def is_need_push(self):
         if self.auto_push and self.protocol in [self.PROTOCOL_SSH, self.PROTOCOL_RDP]:
             return True
@@ -163,7 +162,7 @@ class SystemUser(BaseUser):
 
     @property
     def is_need_test_asset_connective(self):
-        return self.protocol not in self.application_category_protocols
+        return self.protocol in self.ASSET_CATEGORY_PROTOCOLS
 
     def has_special_auth(self, asset=None, username=None):
         if username is None and self.username_same_with_user:
@@ -172,7 +171,7 @@ class SystemUser(BaseUser):
 
     @property
     def can_perm_to_asset(self):
-        return self.protocol not in self.application_category_protocols
+        return self.protocol in self.ASSET_CATEGORY_PROTOCOLS
 
     def _merge_auth(self, other):
         super()._merge_auth(other)
@@ -205,6 +204,17 @@ class SystemUser(BaseUser):
         assets = Asset.objects.filter(id__in=assets_ids)
         return assets
 
+    @classmethod
+    def get_protocol_by_application_type(cls, app_type):
+        from applications.const import ApplicationTypeChoices
+        if app_type in cls.APPLICATION_CATEGORY_PROTOCOLS:
+            protocol = app_type
+        elif app_type in ApplicationTypeChoices.remote_app_types():
+            protocol = cls.PROTOCOL_RDP
+        else:
+            protocol = None
+        return protocol
+
     class Meta:
         ordering = ['name']
         unique_together = [('name', 'org_id')]

apps/assets/serializers/admin_user.py

@@ -3,7 +3,7 @@
 from django.utils.translation import ugettext_lazy as _
 from rest_framework import serializers
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 
 from ..models import Node, AdminUser
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer

apps/assets/serializers/asset.py

@@ -2,7 +2,7 @@
 #
 from rest_framework import serializers
 from django.db.models import F
-
+from django.core.validators import RegexValidator
 from django.utils.translation import ugettext_lazy as _
 
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
@@ -177,6 +177,14 @@ class AssetDisplaySerializer(AssetSerializer):
 class PlatformSerializer(serializers.ModelSerializer):
     meta = serializers.DictField(required=False, allow_null=True)
 
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+        # TODO 修复 drf SlugField RegexValidator bug，之后记得删除
+        validators = self.fields['name'].validators
+        if isinstance(validators[-1], RegexValidator):
+            validators.pop()
+
     class Meta:
         model = Platform
         fields = [
@@ -204,3 +212,6 @@ class AssetTaskSerializer(serializers.Serializer):
     )
     task = serializers.CharField(read_only=True)
     action = serializers.ChoiceField(choices=ACTION_CHOICES, write_only=True)
+    assets = serializers.PrimaryKeyRelatedField(
+        queryset=Asset.objects, required=False, allow_empty=True, many=True
+    )

apps/assets/serializers/asset_user.py

@@ -4,7 +4,7 @@
 from django.utils.translation import ugettext as _
 from rest_framework import serializers
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 from ..models import AuthBook, Asset
 from ..backends import AssetUserManager

apps/assets/serializers/cmd_filter.py

@@ -3,8 +3,7 @@
 import re
 from rest_framework import serializers
 
-from common.fields import ChoiceDisplayField
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from ..models import CommandFilter, CommandFilterRule, SystemUser
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 
@@ -26,7 +25,6 @@ class CommandFilterSerializer(BulkOrgResourceModelSerializer):
 
 
 class CommandFilterRuleSerializer(BulkOrgResourceModelSerializer):
-    # serializer_choice_field = ChoiceDisplayField
     invalid_pattern = re.compile(r'[\.\*\+\[\\\?\{\}\^\$\|\(\)\#\<\>]')
     type_display = serializers.ReadOnlyField(source='get_type_display')
     action_display = serializers.ReadOnlyField(source='get_action_display')

apps/assets/serializers/domain.py

@@ -3,7 +3,7 @@
 from rest_framework import serializers
 from django.utils.translation import ugettext_lazy as _
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 from common.validators import NoSpecialChars
 from ..models import Domain, Gateway

apps/assets/serializers/favorite_asset.py

@@ -4,7 +4,7 @@
 from rest_framework import serializers
 
 from orgs.utils import tmp_to_root_org
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from common.mixins import BulkSerializerMixin
 from ..models import FavoriteAsset
 

apps/assets/serializers/label.py

@@ -2,7 +2,7 @@
 #
 from rest_framework import serializers
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 
 from ..models import Label

apps/assets/serializers/system_user.py

@@ -2,7 +2,7 @@ from rest_framework import serializers
 from django.utils.translation import ugettext_lazy as _
 from django.db.models import Count
 
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from common.mixins.serializers import BulkSerializerMixin
 from common.utils import ssh_pubkey_gen
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer

apps/assets/tasks/asset_connectivity.py

@@ -14,7 +14,7 @@ from .utils import clean_ansible_task_hosts, group_asset_by_platform
 logger = get_logger(__file__)
 __all__ = [
     'test_asset_connectivity_util', 'test_asset_connectivity_manual',
-    'test_node_assets_connectivity_manual',
+    'test_node_assets_connectivity_manual', 'test_assets_connectivity_manual',
 ]
 
 
@@ -82,6 +82,17 @@ def test_asset_connectivity_manual(asset):
         return True, ""
 
 
+@shared_task(queue="ansible")
+def test_assets_connectivity_manual(assets):
+    task_name = _("Test assets connectivity: {}").format([asset.hostname for asset in assets])
+    summary = test_asset_connectivity_util(assets, task_name=task_name)
+
+    if summary.get('dark'):
+        return False, summary['dark']
+    else:
+        return True, ""
+
+
 @shared_task(queue="ansible")
 def test_node_assets_connectivity_manual(node):
     task_name = _("Test if the assets under the node are connectable: {}".format(node.name))

apps/assets/tasks/common.py

@@ -3,10 +3,13 @@
 
 from celery import shared_task
 
+from orgs.utils import tmp_to_root_org
+
 __all__ = ['add_nodes_assets_to_system_users']
 
 
 @shared_task
+@tmp_to_root_org()
 def add_nodes_assets_to_system_users(nodes_keys, system_users):
     from ..models import Node
     assets = Node.get_nodes_all_assets(nodes_keys).values_list('id', flat=True)

apps/assets/tasks/gather_asset_hardware_info.py

@@ -19,6 +19,7 @@ disk_pattern = re.compile(r'^hd|sd|xvd|vd|nv')
 __all__ = [
     'update_assets_hardware_info_util', 'update_asset_hardware_info_manual',
     'update_assets_hardware_info_period',  'update_node_assets_hardware_info_manual',
+    'update_assets_hardware_info_manual',
 ]
 
 
@@ -114,6 +115,12 @@ def update_asset_hardware_info_manual(asset):
     update_assets_hardware_info_util([asset], task_name=task_name)
 
 
+@shared_task(queue="ansible")
+def update_assets_hardware_info_manual(assets):
+    task_name = _("Update assets hardware info: {}").format([asset.hostname for asset in assets])
+    update_assets_hardware_info_util(assets, task_name=task_name)
+
+
 @shared_task(queue="ansible")
 def update_assets_hardware_info_period():
     """

apps/assets/urls/api_urls.py

@@ -36,6 +36,7 @@ urlpatterns = [
     path('assets/<uuid:pk>/gateways/', api.AssetGatewayListApi.as_view(), name='asset-gateway-list'),
     path('assets/<uuid:pk>/platform/', api.AssetPlatformRetrieveApi.as_view(), name='asset-platform-detail'),
     path('assets/<uuid:pk>/tasks/', api.AssetTaskCreateApi.as_view(), name='asset-task-create'),
+    path('assets/tasks/', api.AssetsTaskCreateApi.as_view(), name='assets-task-create'),
 
     path('asset-users/tasks/', api.AssetUserTaskCreateAPI.as_view(), name='asset-user-task-create'),
 

apps/audits/api.py

@@ -25,8 +25,8 @@ class FTPLogViewSet(CreateModelMixin,
     date_range_filter_fields = [
         ('date_start', ('date_from', 'date_to'))
     ]
-    filter_fields = ['user', 'asset', 'system_user', 'filename']
-    search_fields = filter_fields
+    filterset_fields = ['user', 'asset', 'system_user', 'filename']
+    search_fields = filterset_fields
     ordering = ['-date_start']
 
 
@@ -38,7 +38,7 @@ class UserLoginLogViewSet(ListModelMixin, CommonGenericViewSet):
     date_range_filter_fields = [
         ('datetime', ('date_from', 'date_to'))
     ]
-    filter_fields = ['username', 'ip', 'city', 'type', 'status', 'mfa']
+    filterset_fields = ['username', 'ip', 'city', 'type', 'status', 'mfa']
     search_fields =['username', 'ip', 'city']
 
     @staticmethod
@@ -62,7 +62,7 @@ class OperateLogViewSet(ListModelMixin, OrgGenericViewSet):
     date_range_filter_fields = [
         ('datetime', ('date_from', 'date_to'))
     ]
-    filter_fields = ['user', 'action', 'resource_type', 'resource', 'remote_addr']
+    filterset_fields = ['user', 'action', 'resource_type', 'resource', 'remote_addr']
     search_fields = ['resource']
     ordering = ['-datetime']
 
@@ -75,7 +75,7 @@ class PasswordChangeLogViewSet(ListModelMixin, CommonGenericViewSet):
     date_range_filter_fields = [
         ('datetime', ('date_from', 'date_to'))
     ]
-    filter_fields = ['user', 'change_by', 'remote_addr']
+    filterset_fields = ['user', 'change_by', 'remote_addr']
     ordering = ['-datetime']
 
     def get_queryset(self):
@@ -94,7 +94,7 @@ class CommandExecutionViewSet(ListModelMixin, OrgGenericViewSet):
     date_range_filter_fields = [
         ('date_start', ('date_from', 'date_to'))
     ]
-    filter_fields = ['user__name', 'command', 'run_as__name', 'is_finished']
+    filterset_fields = ['user__name', 'command', 'run_as__name', 'is_finished']
     search_fields = ['command', 'user__name', 'run_as__name']
     ordering = ['-date_created']
 
@@ -108,7 +108,7 @@ class CommandExecutionHostRelationViewSet(OrgRelationMixin, OrgBulkModelViewSet)
     serializer_class = CommandExecutionHostsRelationSerializer
     m2m_field = CommandExecution.hosts.field
     permission_classes = [IsOrgAdmin | IsOrgAuditor]
-    filter_fields = [
+    filterset_fields = [
         'id', 'asset', 'commandexecution'
     ]
     search_fields = ('asset__hostname', )

apps/audits/serializers.py

@@ -5,7 +5,7 @@ from rest_framework import serializers
 from django.db.models import F
 
 from common.mixins import BulkSerializerMixin
-from common.serializers import AdaptedBulkListSerializer
+from common.drf.serializers import AdaptedBulkListSerializer
 from terminal.models import Session
 from ops.models import CommandExecution
 from . import models
@@ -86,8 +86,7 @@ class CommandExecutionSerializer(serializers.ModelSerializer):
     @classmethod
     def setup_eager_loading(cls, queryset):
         """ Perform necessary eager loading of data. """
-        queryset = queryset.annotate(user_display=F('user__name'))\
-            .annotate(run_as_display=F('run_as__name'))
+        queryset = queryset.prefetch_related('user', 'run_as', 'hosts')
         return queryset
 
 

apps/audits/signals_handler.py

@@ -125,7 +125,8 @@ def on_audits_log_create(sender, instance=None, **kwargs):
 
 
 def get_login_backend(request):
-    backend = request.session.get(BACKEND_SESSION_KEY, '')
+    backend = request.session.get('auth_backend', '') or request.session.get(BACKEND_SESSION_KEY, '')
+
     backend = backend.rsplit('.', maxsplit=1)[-1]
     if backend in LOGIN_BACKEND:
         return LOGIN_BACKEND[backend]

apps/audits/tasks.py

@@ -40,6 +40,6 @@ def clean_ftp_log_period():
 @register_as_period_task(interval=3600*24)
 @shared_task
 def clean_audits_log_period():
-    clean_audits_log_period()
+    clean_login_log_period()
     clean_operation_log_period()
     clean_ftp_log_period()

apps/authentication/api/auth.py

@@ -4,7 +4,6 @@ import uuid
 
 from django.core.cache import cache
 from django.shortcuts import get_object_or_404
-from rest_framework.permissions import AllowAny
 from rest_framework.response import Response
 from rest_framework.views import APIView
 
@@ -54,12 +53,3 @@ class UserConnectionTokenApi(RootOrgViewMixin, APIView):
             return Response(value)
         else:
             return Response({'user': value['user']})
-
-    def get_permissions(self):
-        if self.request.query_params.get('user-only', None):
-            self.permission_classes = (AllowAny,)
-        return super().get_permissions()
-
-
-
-

apps/authentication/api/login_confirm.py

@@ -45,5 +45,5 @@ class TicketStatusApi(mixins.AuthMixin, APIView):
         ticket = self.get_ticket()
         if ticket:
             request.session.pop('auth_ticket_id', '')
-            ticket.perform_status('closed', request.user)
+            ticket.close(processor=request.user)
         return Response('', status=200)

apps/authentication/mixins.py

@@ -187,12 +187,12 @@ class AuthMixin:
         if not ticket_id:
             ticket = None
         else:
-            ticket = Ticket.origin_objects.get(pk=ticket_id)
+            ticket = Ticket.all().filter(id=ticket_id).first()
         return ticket
 
     def get_ticket_or_create(self, confirm_setting):
         ticket = self.get_ticket()
-        if not ticket or ticket.status == ticket.STATUS.CLOSED:
+        if not ticket or ticket.status_closed:
             ticket = confirm_setting.create_confirm_ticket(self.request)
             self.request.session['auth_ticket_id'] = str(ticket.id)
         return ticket
@@ -201,12 +201,16 @@ class AuthMixin:
         ticket = self.get_ticket()
         if not ticket:
             raise errors.LoginConfirmOtherError('', "Not found")
-        if ticket.status == ticket.STATUS.OPEN:
+        if ticket.status_open:
             raise errors.LoginConfirmWaitError(ticket.id)
-        elif ticket.action == ticket.ACTION.APPROVE:
+        elif ticket.action_approve:
             self.request.session["auth_confirm"] = "1"
             return
-        elif ticket.action == ticket.ACTION.REJECT:
+        elif ticket.action_reject:
+            raise errors.LoginConfirmOtherError(
+                ticket.id, ticket.get_action_display()
+            )
+        elif ticket.action_close:
             raise errors.LoginConfirmOtherError(
                 ticket.id, ticket.get_action_display()
             )

apps/authentication/models.py

@@ -49,29 +49,36 @@ class LoginConfirmSetting(CommonModelMixin):
     def get_user_confirm_setting(cls, user):
         return get_object_or_none(cls, user=user)
 
-    def create_confirm_ticket(self, request=None):
-        from tickets.models import Ticket
-        title = _('Login confirm') + ' {}'.format(self.user)
+    @staticmethod
+    def construct_confirm_ticket_meta(request=None):
         if request:
-            remote_addr = get_request_ip(request)
-            city = get_ip_city(remote_addr)
-            datetime = timezone.now().strftime('%Y-%m-%d %H:%M:%S')
-            body = __("{user_key}: {username}<br>"
-                      "IP: {ip}<br>"
-                      "{city_key}: {city}<br>"
-                      "{date_key}: {date}<br>").format(
-                user_key=__("User"), username=self.user,
-                ip=remote_addr, city_key=_("City"), city=city,
-                date_key=__("Datetime"), date=datetime
-            )
+            login_ip = get_request_ip(request)
         else:
-            body = ''
-        reviewer = self.reviewers.all()
-        ticket = Ticket.objects.create(
-            user=self.user, title=title, body=body,
-            type=Ticket.TYPE.LOGIN_CONFIRM,
-        )
-        ticket.assignees.set(reviewer)
+            login_ip = ''
+        login_ip = login_ip or '0.0.0.0'
+        login_city = get_ip_city(login_ip)
+        login_datetime = timezone.now().strftime('%Y-%m-%d %H:%M:%S')
+        ticket_meta = {
+            'apply_login_ip': login_ip,
+            'apply_login_city': login_city,
+            'apply_login_datetime': login_datetime,
+        }
+        return ticket_meta
+
+    def create_confirm_ticket(self, request=None):
+        from tickets import const
+        from tickets.models import Ticket
+        ticket_title = _('Login confirm') + ' {}'.format(self.user)
+        ticket_meta = self.construct_confirm_ticket_meta(request)
+        ticket_assignees = self.reviewers.all()
+        data = {
+            'title': ticket_title,
+            'type': const.TicketTypeChoices.login_confirm.value,
+            'meta': ticket_meta,
+        }
+        ticket = Ticket.objects.create(**data)
+        ticket.assignees.set(ticket_assignees)
+        ticket.open(self.user)
         return ticket
 
     def __str__(self):

apps/authentication/templates/authentication/login.html

@@ -1,82 +1,179 @@
-{% extends '_base_only_msg_content.html' %}
 {% load static %}
 {% load i18n %}
+<!DOCTYPE html>
+<html>
+<!--/*@thymesVar id="LoginConstants" type="com.fit2cloud.support.common.constants.LoginConstants"*/-->
+<!--/*@thymesVar id="message" type="java.lang.String"*/-->
+<head>
+    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
+        <link rel="shortcut icon" href="{{ FAVICON_URL }}" type="image/x-icon">
+    <title>
+        {{ JMS_TITLE }}
+    </title>
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <!-- Stylesheets -->
+    <link href="{% static 'css/bootstrap.min.css' %}" rel="stylesheet">
+    <link href="{% static 'css/font-awesome.min.css' %}" rel="stylesheet">
+    <link href="{% static 'css/bootstrap-style.css' %}" rel="stylesheet">
+    <link href="{% static 'css/login-style.css' %}" rel="stylesheet">
 
-{% block content_title %}
-    {% trans 'Login' %}
-{% endblock %}
+    <!-- scripts -->
+    <script src="{% static 'js/jquery-3.1.1.min.js' %}"></script>
+    <script src="{% static 'js/plugins/sweetalert/sweetalert.min.js' %}"></script>
+    <script src="{% static 'js/bootstrap.min.js' %}"></script>
+    <script src="{% static 'js/plugins/datatables/datatables.min.js' %}"></script>
 
-{% block content %}
-    <form id="form" class="m-t" role="form" method="post" action="">
-        {% csrf_token %}
-        {% if form.non_field_errors %}
-            <div style="line-height: 17px;">
-                <p class="red-fonts">{{ form.non_field_errors.as_text }}</p>
+    <style>
+
+        .box-1{
+            height: 472px;
+            width: 984px;
+            margin-right: auto;
+            margin-left: auto;
+            margin-top: calc((100vh - 470px)/2);
+
+        }
+        .box-2{
+            height: 100%;
+            width: 50%;
+            float: right;
+        }
+        .box-3{
+            text-align: center;
+            background-color: white;
+            height: 100%;
+            width: 50%;
+        }
+        .captcha {
+            float: right;
+        }
+
+        .red-fonts {
+            color: red;
+        }
+
+        .field-error {
+            text-align: left;
+        }
+
+    </style>
+</head>
+
+<body style="height: 100%;font-size: 13px">
+    <div>
+        <div class="box-1">
+            <div class="box-2">
+                <img src="{{ LOGIN_IMAGE_URL }}" style="height: 100%; width: 100%"/>
             </div>
-        {% elif form.errors.captcha %}
-            <p class="red-fonts">{% trans 'Captcha invalid' %}</p>
-        {% endif %}
-
-        <div class="form-group">
-            <input type="text" class="form-control" name="{{ form.username.html_name }}" placeholder="{% trans 'Username' %}" required="" value="{% if form.username.value %}{{ form.username.value }}{% endif %}">
-            {% if form.errors.username %}
-                <div class="help-block field-error">
-                    <p class="red-fonts">{{ form.errors.username.as_text }}</p>
-                </div>
-            {% endif %}
-        </div>
-        <div class="form-group">
-            <input type="password" class="form-control" id="password" placeholder="{% trans 'Password' %}" required="">
-            <input id="password-hidden" type="text" style="display:none" name="{{ form.password.html_name }}">
-            {% if form.errors.password %}
-                <div class="help-block field-error">
-                    <p class="red-fonts">{{ form.errors.password.as_text }}</p>
-                </div>
-            {% endif %}
-        </div>
-        {% if form.challenge %}
-            <div class="form-group">
-                <input type="challenge" class="form-control" id="challenge" name="{{ form.challenge.html_name }}" placeholder="{% trans 'MFA code' %}" >
-                {% if form.errors.challenge %}
-                    <div class="help-block field-error">
-                        <p class="red-fonts">{{ form.errors.challenge.as_text }}</p>
+            <div class="box-3">
+                <div style="background-color: white">
+                    {% if form.challenge %}
+                        <div style="margin-top: 20px;padding-top: 30px;padding-left: 20px;padding-right: 20px;height: 60px">
+                    {% else %}
+                        <div style="margin-top: 20px;padding-top: 40px;padding-left: 20px;padding-right: 20px;height: 80px">
+                    {% endif %}
+                            <span style="font-size: 21px;font-weight:400;color: #151515;letter-spacing: 0;">{{ JMS_TITLE }}</span>
+                        </div>
+                    <div style="font-size: 12px;color: #999999;letter-spacing: 0;line-height: 18px;margin-top: 18px">
+                        {% trans 'Welcome back, please enter username and password to login' %}
                     </div>
-                {% endif %}
-            </div>
-        {% endif %}
-        <div>
-            {{ form.captcha }}
-        </div>
-        <button type="submit" class="btn btn-primary block full-width m-b" onclick="doLogin();return false;">{% trans 'Login' %}</button>
+                    <div style="margin-bottom: 0px">
+                        <div>
+                            <div class="col-md-1"></div>
+                            <div class="contact-form col-md-10" style="margin-top: 0px;height: 35px">
+                                <form id="contact-form" action="" method="post" role="form" novalidate="novalidate">
+                                    {% csrf_token %}
+                                    {% if form.non_field_errors %}
+                                        {% if form.challenge %}
+                                            <div style="height: 50px;color: red;line-height: 17px;">
+                                        {% else %}
+                                            <div style="height: 70px;color: red;line-height: 17px;">
+                                        {% endif %}
+                                            <p class="red-fonts">{{ form.non_field_errors.as_text }}</p>
+                                        </div>
+                                    {% elif form.errors.captcha %}
+                                        <p class="red-fonts">{% trans 'Captcha invalid' %}</p>
+                                    {% else %}
+                                        <div style="height: 50px"></div>
+                                    {% endif %}
 
-        {% if demo_mode %}
-            <p class="text-muted font-bold" style="color: red">
-                Demo账号: admin 密码: admin
-            </p>
-        {% endif %}
+                                    <div class="form-group">
+                                        <input type="text" class="form-control" name="{{ form.username.html_name }}" placeholder="{% trans 'Username' %}" required="" value="{% if form.username.value %}{{ form.username.value }}{% endif %}" style="height: 35px">
+                                        {% if form.errors.username %}
+                                            <div class="help-block field-error">
+                                                <p class="red-fonts">{{ form.errors.username.as_text }}</p>
+                                            </div>
+                                        {% endif %}
+                                    </div>
+                                    <div class="form-group">
+                                        <input type="password" class="form-control" id="password" placeholder="{% trans 'Password' %}" required="">
+                                        <input id="password-hidden" type="text" style="display:none" name="{{ form.password.html_name }}">
+                                        {% if form.errors.password %}
+                                            <div class="help-block field-error">
+                                                <p class="red-fonts">{{ form.errors.password.as_text }}</p>
+                                            </div>
+                                        {% endif %}
+                                    </div>
+                                    {% if form.challenge %}
+                                        <div class="form-group">
+                                            <input type="challenge" class="form-control" id="challenge" name="{{ form.challenge.html_name }}" placeholder="{% trans 'MFA code' %}" >
+                                            {% if form.errors.challenge %}
+                                                <div class="help-block field-error">
+                                                    <p class="red-fonts">{{ form.errors.challenge.as_text }}</p>
+                                                </div>
+                                            {% endif %}
+                                        </div>
+                                    {% endif %}
+                                    {% if form.captcha %}
+                                    <div class="form-group" style="height: 50px;margin-bottom: 0;font-size: 13px">
+                                        {{ form.captcha }}
+                                    </div>
+                                    {% else %}
+                                    <div class="form-group" style="height: 25px;margin-bottom: 0;font-size: 13px"></div>
+                                    {% endif %}
+                                    <div class="form-group" style="margin-top: 10px">
+                                        <button type="submit" class="btn btn-transparent" onclick="doLogin();return false;">{% trans 'Login' %}</button>
+                                    </div>
 
-        <div class="text-muted text-center">
-            <div>
-                <a id="forgot_password" href="#">
-                    <small>{% trans 'Forgot password' %}?</small>
-                </a>
+                                    <div>
+                                        {% if AUTH_OPENID or AUTH_CAS %}
+                                            <div class="hr-line-dashed"></div>
+                                            <div style="display: inline-block; float: left">
+                                                <b class="text-muted text-left" style="margin-right: 10px">{% trans "More login options" %}</b>
+                                                {% if AUTH_OPENID %}
+                                                    <a href="{% url 'authentication:openid:login' %}">
+                                                        <i class="fa fa-openid"></i> {% trans 'OpenID' %}
+                                                    </a>
+                                                {% endif %}
+                                                {% if AUTH_CAS %}
+                                                    <a href="{% url 'authentication:cas:cas-login' %}">
+                                                        <i class="fa"><img src="{{ LOGIN_CAS_LOGO_URL }}" height="13" width="13"></i> {% trans 'CAS' %}
+                                                    </a>
+                                                {% endif %}
+                                            </div>
+                                            <div class="text-center" style="display: inline-block; float: right">
+                                        {% else %}
+                                            <div class="text-center" style="display: inline-block;">
+                                        {% endif %}
+                                            <a id="forgot_password" href="{% url 'authentication:forgot-password' %}">
+                                                <small>{% trans 'Forgot password' %}?</small>
+                                            </a>
+                                        </div>
+                                    </div>
+                                </form>
+                            </div>
+                            <div class="col-md-1"></div>
+                        </div>
+                    </div>
+                </div>
             </div>
         </div>
+    </div>
+    </div>
 
-        {% if AUTH_OPENID %}
-            <div class="hr-line-dashed"></div>
-            <p class="text-muted text-center">{% trans "More login options" %}</p>
-            <div>
-                <button type="button" class="btn btn-default btn-sm btn-block" onclick="location.href='{% url 'authentication:openid:login' %}'">
-                    <i class="fa fa-openid"></i>
-                    {% trans 'OpenID' %}
-                </button>
-            </div>
-        {% endif %}
-
-    </form>
-    <script type="text/javascript" src="/static/js/plugins/jsencrypt/jsencrypt.min.js"></script>
-    <script>
+</body>
+<script type="text/javascript" src="/static/js/plugins/jsencrypt/jsencrypt.min.js"></script>
+<script>
     function encryptLoginPassword(password, rsaPublicKey){
         var jsencrypt = new JSEncrypt(); //加密对象
         jsencrypt.setPublicKey(rsaPublicKey); // 设置密钥
@@ -88,19 +185,11 @@
         var password =$('#password').val(); //明文密码
         var passwordEncrypted = encryptLoginPassword(password, rsaPublicKey)
         $('#password-hidden').val(passwordEncrypted); //返回给密码输入input
-        $('#form').submit();//post提交
+        $('#contact-form').submit();//post提交
     }
 
-    var authDB = '{{ AUTH_DB }}';
-    var forgotPasswordUrl = "{% url 'authentication:forgot-password' %}";
     $(document).ready(function () {
-    }).on('click', '#forgot_password', function () {
-        if (authDB === 'True'){
-            window.open(forgotPasswordUrl, "_blank")
-        }
-        else{
-            alert("{% trans 'You are using another authentication server, please contact your administrator' %}")
-        }
     })
-    </script>
-{% endblock %}
+</script>
+</html>
+

apps/authentication/templates/authentication/xpack_login.html

@@ -1,179 +0,0 @@
-{% load static %}
-{% load i18n %}
-<!DOCTYPE html>
-<html>
-<!--/*@thymesVar id="LoginConstants" type="com.fit2cloud.support.common.constants.LoginConstants"*/-->
-<!--/*@thymesVar id="message" type="java.lang.String"*/-->
-<head>
-    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
-        <link rel="shortcut icon" href="{{ FAVICON_URL }}" type="image/x-icon">
-    <title>
-        {{ JMS_TITLE }}
-    </title>
-    <meta name="viewport" content="width=device-width, initial-scale=1.0">
-    <!-- Stylesheets -->
-    <link href="{% static 'css/bootstrap.min.css' %}" rel="stylesheet">
-    <link href="{% static 'css/font-awesome.min.css' %}" rel="stylesheet">
-    <link href="{% static 'css/bootstrap-style.css' %}" rel="stylesheet">
-    <link href="{% static 'css/login-style.css' %}" rel="stylesheet">
-
-    <!-- scripts -->
-    <script src="{% static 'js/jquery-3.1.1.min.js' %}"></script>
-    <script src="{% static 'js/plugins/sweetalert/sweetalert.min.js' %}"></script>
-    <script src="{% static 'js/bootstrap.min.js' %}"></script>
-    <script src="{% static 'js/plugins/datatables/datatables.min.js' %}"></script>
-
-    <style>
-
-        .box-1{
-            height: 472px;
-            width: 984px;
-            margin-right: auto;
-            margin-left: auto;
-            margin-top: calc((100vh - 470px)/2);
-
-        }
-        .box-2{
-            height: 100%;
-            width: 50%;
-            float: right;
-        }
-        .box-3{
-            text-align: center;
-            background-color: white;
-            height: 100%;
-            width: 50%;
-        }
-        .captcha {
-            float: right;
-        }
-
-        .red-fonts {
-            color: red;
-        }
-
-        .field-error {
-            text-align: left;
-        }
-
-    </style>
-</head>
-
-<body style="height: 100%;font-size: 13px">
-    <div>
-        <div class="box-1">
-            <div class="box-2">
-                <img src="{{ LOGIN_IMAGE_URL }}" style="height: 100%; width: 100%"/>
-            </div>
-            <div class="box-3">
-                <div style="background-color: white">
-                    {% if form.challenge %}
-                        <div style="margin-top: 20px;padding-top: 30px;padding-left: 20px;padding-right: 20px;height: 60px">
-                    {% else %}
-                        <div style="margin-top: 20px;padding-top: 40px;padding-left: 20px;padding-right: 20px;height: 80px">
-                    {% endif %}
-                            <span style="font-size: 21px;font-weight:400;color: #151515;letter-spacing: 0;">{{ JMS_TITLE }}</span>
-                        </div>
-                    <div style="font-size: 12px;color: #999999;letter-spacing: 0;line-height: 18px;margin-top: 18px">
-                        {% trans 'Welcome back, please enter username and password to login' %}
-                    </div>
-                    <div style="margin-bottom: 0px">
-                        <div>
-                            <div class="col-md-1"></div>
-                            <div class="contact-form col-md-10" style="margin-top: 0px;height: 35px">
-                                <form id="contact-form" action="" method="post" role="form" novalidate="novalidate">
-                                    {% csrf_token %}
-                                    {% if form.non_field_errors %}
-                                        {% if form.challenge %}
-                                            <div style="height: 50px;color: red;line-height: 17px;">
-                                        {% else %}
-                                            <div style="height: 70px;color: red;line-height: 17px;">
-                                        {% endif %}
-                                            <p class="red-fonts">{{ form.non_field_errors.as_text }}</p>
-                                        </div>
-                                    {% elif form.errors.captcha %}
-                                        <p class="red-fonts">{% trans 'Captcha invalid' %}</p>
-                                    {% else %}
-                                        <div style="height: 50px"></div>
-                                    {% endif %}
-
-                                    <div class="form-group">
-                                        <input type="text" class="form-control" name="{{ form.username.html_name }}" placeholder="{% trans 'Username' %}" required="" value="{% if form.username.value %}{{ form.username.value }}{% endif %}" style="height: 35px">
-                                        {% if form.errors.username %}
-                                            <div class="help-block field-error">
-                                                <p class="red-fonts">{{ form.errors.username.as_text }}</p>
-                                            </div>
-                                        {% endif %}
-                                    </div>
-                                    <div class="form-group">
-                                        <input type="password" class="form-control" id="password" placeholder="{% trans 'Password' %}" required="">
-                                        <input id="password-hidden" type="text" style="display:none" name="{{ form.password.html_name }}">
-                                        {% if form.errors.password %}
-                                            <div class="help-block field-error">
-                                                <p class="red-fonts">{{ form.errors.password.as_text }}</p>
-                                            </div>
-                                        {% endif %}
-                                    </div>
-                                    {% if form.challenge %}
-                                        <div class="form-group">
-                                            <input type="challenge" class="form-control" id="challenge" name="{{ form.challenge.html_name }}" placeholder="{% trans 'MFA code' %}" >
-                                            {% if form.errors.challenge %}
-                                                <div class="help-block field-error">
-                                                    <p class="red-fonts">{{ form.errors.challenge.as_text }}</p>
-                                                </div>
-                                            {% endif %}
-                                        </div>
-                                    {% endif %}
-                                    <div class="form-group" style="height: 50px;margin-bottom: 0;font-size: 13px">
-                                        {{ form.captcha }}
-                                    </div>
-                                    <div class="form-group" style="margin-top: 10px">
-                                        <button type="submit" class="btn btn-transparent" onclick="doLogin();return false;">{% trans 'Login' %}</button>
-                                    </div>
-                                    <div style="text-align: center">
-                                        <a id="forgot_password" href="#">
-                                            <small>{% trans 'Forgot password' %}?</small>
-                                        </a>
-                                    </div>
-                                </form>
-                            </div>
-                            <div class="col-md-1"></div>
-                        </div>
-                    </div>
-                </div>
-            </div>
-        </div>
-    </div>
-    </div>
-
-</body>
-<script type="text/javascript" src="/static/js/plugins/jsencrypt/jsencrypt.min.js"></script>
-<script>
-    function encryptLoginPassword(password, rsaPublicKey){
-        var jsencrypt = new JSEncrypt(); //加密对象
-        jsencrypt.setPublicKey(rsaPublicKey); // 设置密钥
-        return jsencrypt.encrypt(password); //加密
-    }
-    function doLogin() {
-        //公钥加密
-        var rsaPublicKey = "{{ rsa_public_key }}"
-        var password =$('#password').val(); //明文密码
-        var passwordEncrypted = encryptLoginPassword(password, rsaPublicKey)
-        $('#password-hidden').val(passwordEncrypted); //返回给密码输入input
-        $('#contact-form').submit();//post提交
-    }
-
-    var authDB = '{{ AUTH_DB }}';
-    var forgotPasswordUrl = "{% url 'authentication:forgot-password' %}";
-    $(document).ready(function () {
-    }).on('click', '#forgot_password', function () {
-        if (authDB === 'True'){
-            window.open(forgotPasswordUrl, "_blank")
-        }
-        else{
-            alert("{% trans 'You are using another authentication server, please contact your administrator' %}")
-        }
-    })
-</script>
-</html>
-

apps/authentication/views/login.py

@@ -19,7 +19,6 @@ from django.conf import settings
 from django.urls import reverse_lazy
 from django.contrib.auth import BACKEND_SESSION_KEY
 
-from common.const.front_urls import TICKET_DETAIL
 from common.utils import get_request_ip, get_object_or_none
 from users.utils import (
     redirect_user_first_login_or_index
@@ -42,42 +41,13 @@ __all__ = [
 class UserLoginView(mixins.AuthMixin, FormView):
     key_prefix_captcha = "_LOGIN_INVALID_{}"
     redirect_field_name = 'next'
-
-    def get_template_names(self):
-        template_name = 'authentication/login.html'
-        if not settings.XPACK_ENABLED:
-            return template_name
-
-        from xpack.plugins.license.models import License
-        if not License.has_valid_license():
-            return template_name
-
-        template_name = 'authentication/xpack_login.html'
-        return template_name
-
-    def get_redirect_url_if_need(self, request):
-        redirect_url = ''
-        # show jumpserver login page if request http://{JUMP-SERVER}/?admin=1
-        if self.request.GET.get("admin", 0):
-            return None
-        if settings.AUTH_OPENID:
-            redirect_url = reverse(settings.AUTH_OPENID_AUTH_LOGIN_URL_NAME)
-        elif settings.AUTH_CAS:
-            redirect_url = reverse(settings.CAS_LOGIN_URL_NAME)
-
-        if redirect_url:
-            query_string = request.GET.urlencode()
-            redirect_url = "{}?{}".format(redirect_url, query_string)
-        return redirect_url
+    template_name = 'authentication/login.html'
 
     def get(self, request, *args, **kwargs):
         if request.user.is_staff:
             return redirect(redirect_user_first_login_or_index(
                 request, self.redirect_field_name)
             )
-        redirect_url = self.get_redirect_url_if_need(request)
-        if redirect_url:
-            return redirect(redirect_url)
         request.session.set_test_cookie()
         return super().get(request, *args, **kwargs)
 
@@ -132,8 +102,8 @@ class UserLoginView(mixins.AuthMixin, FormView):
         context = {
             'demo_mode': os.environ.get("DEMO_MODE"),
             'AUTH_OPENID': settings.AUTH_OPENID,
+            'AUTH_CAS': settings.AUTH_CAS,
             'rsa_public_key': rsa_public_key,
-            'AUTH_DB': settings.AUTH_DB
         }
         kwargs.update(context)
         return super().get_context_data(**kwargs)
@@ -181,6 +151,7 @@ class UserLoginWaitConfirmView(TemplateView):
 
     def get_context_data(self, **kwargs):
         from tickets.models import Ticket
+        from tickets.const import TICKET_DETAIL_URL
         ticket_id = self.request.session.get("auth_ticket_id")
         if not ticket_id:
             ticket = None
@@ -189,7 +160,7 @@ class UserLoginWaitConfirmView(TemplateView):
         context = super().get_context_data(**kwargs)
         if ticket:
             timestamp_created = datetime.datetime.timestamp(ticket.date_created)
-            ticket_detail_url = TICKET_DETAIL.format(id=ticket_id)
+            ticket_detail_url = TICKET_DETAIL_URL.format(id=ticket_id)
             msg = _("""Wait for <b>{}</b> confirm, You also can copy link to her/him <br/>
                   Don't close this page""").format(ticket.assignees_display)
         else:

apps/common/cache.py

@@ -0,0 +1,187 @@
+import json
+from django.core.cache import cache
+
+from common.utils.lock import DistributedLock
+from common.utils import lazyproperty
+from common.utils import get_logger
+
+logger = get_logger(__file__)
+
+
+class CacheFieldBase:
+    field_type = str
+
+    def __init__(self, queryset=None, compute_func_name=None):
+        assert None in (queryset, compute_func_name), f'queryset and compute_func_name can only have one'
+        self.compute_func_name = compute_func_name
+        self.queryset = queryset
+
+
+class CharField(CacheFieldBase):
+    field_type = str
+
+
+class IntegerField(CacheFieldBase):
+    field_type = int
+
+
+class CacheBase(type):
+    def __new__(cls, name, bases, attrs: dict):
+        to_update = {}
+        field_desc_mapper = {}
+
+        for k, v in attrs.items():
+            if isinstance(v, CacheFieldBase):
+                desc = CacheValueDesc(k, v)
+                to_update[k] = desc
+                field_desc_mapper[k] = desc
+
+        attrs.update(to_update)
+        attrs['field_desc_mapper'] = field_desc_mapper
+        return type.__new__(cls, name, bases, attrs)
+
+
+class Cache(metaclass=CacheBase):
+    field_desc_mapper: dict
+    timeout = None
+
+    def __init__(self):
+        self._data = None
+
+    @lazyproperty
+    def key_suffix(self):
+        return self.get_key_suffix()
+
+    @property
+    def key_prefix(self):
+        clz = self.__class__
+        return f'cache.{clz.__module__}.{clz.__name__}'
+
+    @property
+    def key(self):
+        return f'{self.key_prefix}.{self.key_suffix}'
+
+    @property
+    def data(self):
+        if self._data is None:
+            data = self.get_data()
+            if data is None:
+                # 缓存中没有数据时，去数据库获取
+                self.compute_and_set_all_data()
+        return self._data
+
+    def get_data(self) -> dict:
+        data = cache.get(self.key)
+        logger.debug(f'CACHE: get {self.key} = {data}')
+        if data is not None:
+            data = json.loads(data)
+            self._data = data
+        return data
+
+    def set_data(self, data):
+        self._data = data
+        to_json = json.dumps(data)
+        logger.info(f'CACHE: set {self.key} = {to_json}, timeout={self.timeout}')
+        cache.set(self.key, to_json, timeout=self.timeout)
+
+    def compute_data(self, *fields):
+        field_descs = []
+        if not fields:
+            field_descs = self.field_desc_mapper.values()
+        else:
+            for field in fields:
+                assert field in self.field_desc_mapper, f'{field} is not a valid field'
+                field_descs.append(self.field_desc_mapper[field])
+        data = {
+            field_desc.field_name: field_desc.compute_value(self)
+            for field_desc in field_descs
+        }
+        return data
+
+    def compute_and_set_all_data(self, computed_data: dict = None):
+        """
+        TODO 怎样防止并发更新全部数据，浪费数据库资源
+        """
+        uncomputed_keys = ()
+        if computed_data:
+            computed_keys = computed_data.keys()
+            all_keys = self.field_desc_mapper.keys()
+            uncomputed_keys = all_keys - computed_keys
+        else:
+            computed_data = {}
+        data = self.compute_data(*uncomputed_keys)
+        data.update(computed_data)
+        self.set_data(data)
+        return data
+
+    def refresh_part_data_with_lock(self, refresh_data):
+        with DistributedLock(name=f'{self.key}.refresh'):
+            data = self.get_data()
+            if data is not None:
+                data.update(refresh_data)
+                self.set_data(data)
+                return data
+
+    def refresh(self, *fields):
+        if not fields:
+            # 没有指定 field 要刷新所有的值
+            self.compute_and_set_all_data()
+            return
+
+        data = self.get_data()
+        if data is None:
+            # 缓存中没有数据，设置所有的值
+            self.compute_and_set_all_data()
+            return
+
+        refresh_data = self.compute_data(*fields)
+        if not self.refresh_part_data_with_lock(refresh_data):
+            # 刷新部分失败，缓存中没有数据，更新所有的值
+            self.compute_and_set_all_data(refresh_data)
+            return
+
+    def get_key_suffix(self):
+        raise NotImplementedError
+
+    def reload(self):
+        self._data = None
+
+    def delete(self):
+        self._data = None
+        logger.info(f'CACHE: delete {self.key}')
+        cache.delete(self.key)
+
+
+class CacheValueDesc:
+    def __init__(self, field_name, field_type: CacheFieldBase):
+        self.field_name = field_name
+        self.field_type = field_type
+        self._data = None
+
+    def __repr__(self):
+        clz = self.__class__
+        return f'<{clz.__name__} {self.field_name} {self.field_type}>'
+
+    def __get__(self, instance: Cache, owner):
+        if instance is None:
+            return self
+        if self.field_name not in instance.data:
+            instance.refresh(self.field_name)
+        value = instance.data[self.field_name]
+        return value
+
+    def compute_value(self, instance: Cache):
+        if self.field_type.queryset is not None:
+            new_value = self.field_type.queryset.count()
+        else:
+            compute_func_name = self.field_type.compute_func_name
+            if not compute_func_name:
+                compute_func_name = f'compute_{self.field_name}'
+            compute_func = getattr(instance, compute_func_name, None)
+            assert compute_func is not None, \
+                f'Define `{compute_func_name}` method in {instance.__class__}'
+            new_value = compute_func()
+
+        new_value = self.field_type.field_type(new_value)
+        logger.info(f'CACHE: compute {instance.key}.{self.field_name} = {new_value}')
+        return new_value

apps/common/const/choices.py

@@ -1,7 +1,3 @@
-from django.utils.translation import ugettext_lazy as _
-
-from common.db.models import ChoiceSet
-
 
 ADMIN = 'Admin'
 USER = 'User'

apps/common/const/front_urls.py

@@ -1,2 +0,0 @@
-
-TICKET_DETAIL = '/ui/#/tickets/tickets/{id}'

apps/common/const/signals.py

@@ -12,3 +12,6 @@ PRE_REMOVE = 'pre_remove'
 POST_REMOVE = 'post_remove'
 PRE_CLEAR = 'pre_clear'
 POST_CLEAR = 'post_clear'
+
+POST_PREFIX = 'post'
+PRE_PREFIX = 'pre'

apps/common/db/utils.py

@@ -35,6 +35,6 @@ def get_objects(model, pks):
     if len(objs) != len(pks):
         pks = set(pks)
         exists_pks = {o.id for o in objs}
-        not_found_pks = ','.join(pks - exists_pks)
+        not_found_pks = pks - exists_pks
         logger.error(f'DoesNotExist: <{model.__name__}: {not_found_pks}>')
     return objs

apps/common/drf/fields.py

@@ -1,43 +1,25 @@
-from uuid import UUID
+# -*- coding: utf-8 -*-
+#
 
-from rest_framework.fields import get_attribute
-from rest_framework.relations import ManyRelatedField, PrimaryKeyRelatedField, MANY_RELATION_KWARGS
+from rest_framework import serializers
 
 
-class GroupConcatedManyRelatedField(ManyRelatedField):
-    def get_attribute(self, instance):
-        if hasattr(instance, 'pk') and instance.pk is None:
-            return []
-
-        attr = self.source_attrs[-1]
-
-        # `gc` 是 `GroupConcat` 的缩写
-        gc_attr = f'gc_{attr}'
-        if hasattr(instance, gc_attr):
-            gc_value = getattr(instance, gc_attr)
-            if isinstance(gc_value, str):
-                return [UUID(pk) for pk in set(gc_value.split(','))]
-            else:
-                return ''
-
-        relationship = get_attribute(instance, self.source_attrs)
-        return relationship.all() if hasattr(relationship, 'all') else relationship
+__all__ = [
+    'ReadableHiddenField',
+]
 
 
-class GroupConcatedPrimaryKeyRelatedField(PrimaryKeyRelatedField):
-    @classmethod
-    def many_init(cls, *args, **kwargs):
-        list_kwargs = {'child_relation': cls(*args, **kwargs)}
-        for key in kwargs:
-            if key in MANY_RELATION_KWARGS:
-                list_kwargs[key] = kwargs[key]
-        return GroupConcatedManyRelatedField(**list_kwargs)
+# ReadableHiddenField
+# -------------------
+
+
+class ReadableHiddenField(serializers.HiddenField):
+    """ 可读的 HiddenField """
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        self.write_only = False
 
     def to_representation(self, value):
-        if self.pk_field is not None:
-            return self.pk_field.to_representation(value.pk)
-
-        if hasattr(value, 'pk'):
-            return value.pk
-        else:
-            return value
+        if hasattr(value, 'id'):
+            return getattr(value, 'id')
+        return value

apps/common/drf/metadata.py

@@ -73,9 +73,8 @@ class SimpleMetadataWithFilters(SimpleMetadata):
         elif getattr(field, 'fields', None):
             field_info['children'] = self.get_serializer_info(field)
 
-        if (not field_info.get('read_only') and
-            not isinstance(field, (serializers.RelatedField, serializers.ManyRelatedField)) and
-                hasattr(field, 'choices')):
+        if not isinstance(field, (serializers.RelatedField, serializers.ManyRelatedField)) \
+                and hasattr(field, 'choices'):
             field_info['choices'] = [
                 {
                     'value': choice_value,
@@ -92,6 +91,13 @@ class SimpleMetadataWithFilters(SimpleMetadata):
             fields = view.get_filter_fields(request)
         elif hasattr(view, 'filter_fields'):
             fields = view.filter_fields
+        elif hasattr(view, 'filterset_fields'):
+            fields = view.filterset_fields
+        elif hasattr(view, 'get_filterset_fields'):
+            fields = view.get_filterset_fields(request)
+
+        if isinstance(fields, dict):
+            fields = list(fields.keys())
         return fields
 
     def get_ordering_fields(self, request, view):
@@ -104,12 +110,12 @@ class SimpleMetadataWithFilters(SimpleMetadata):
 
     def determine_metadata(self, request, view):
         metadata = super(SimpleMetadataWithFilters, self).determine_metadata(request, view)
-        filter_fields = self.get_filters_fields(request, view)
+        filterset_fields = self.get_filters_fields(request, view)
         order_fields = self.get_ordering_fields(request, view)
 
         meta_get = metadata.get("actions", {}).get("GET", {})
         for k, v in meta_get.items():
-            if k in filter_fields:
+            if k in filterset_fields:
                 v["filter"] = True
             if k in order_fields:
                 v["order"] = True

apps/common/drf/parsers/base.py

@@ -1,6 +1,7 @@
 import abc
 import json
 import codecs
+from rest_framework import serializers
 from django.utils.translation import ugettext_lazy as _
 from rest_framework.parsers import BaseParser
 from rest_framework import status
@@ -83,14 +84,17 @@ class BaseFileParser(BaseParser):
             new_row.append(col)
         return new_row
 
-    @staticmethod
-    def process_row_data(row_data):
+    def process_row_data(self, row_data):
         """
         构建json数据后的行数据处理
         """
         new_row_data = {}
+        serializer_fields = self.serializer_cls().fields
         for k, v in row_data.items():
             if isinstance(v, list) or isinstance(v, dict) or isinstance(v, str) and k.strip() and v.strip():
+                # 解决类似disk_info为字符串的'{}'的问题
+                if not isinstance(v, str) and isinstance(serializer_fields[k], serializers.CharField):
+                    v = str(v)
                 new_row_data[k] = v
         return new_row_data
 

apps/common/drf/serializers.py

@@ -1,12 +1,70 @@
+import copy
+from rest_framework import serializers
 from rest_framework.serializers import Serializer
 from rest_framework.serializers import ModelSerializer
-from rest_framework import serializers
 from rest_framework_bulk.serializers import BulkListSerializer
 
-from common.mixins.serializers import BulkSerializerMixin
 from common.mixins import BulkListSerializerMixin
+from django.utils.functional import cached_property
+from rest_framework.utils.serializer_helpers import BindingDict
+from common.mixins.serializers import BulkSerializerMixin
 
-__all__ = ['EmptySerializer', 'BulkModelSerializer']
+__all__ = [
+    'MethodSerializer',
+    'EmptySerializer', 'BulkModelSerializer', 'AdaptedBulkListSerializer', 'CeleryTaskSerializer'
+]
+
+
+# MethodSerializer
+# ----------------
+
+
+class MethodSerializer(serializers.Serializer):
+
+    def __init__(self, method_name=None, **kwargs):
+        self.method_name = method_name
+        super().__init__(**kwargs)
+
+    class Meta:
+        # 生成swagger时使用
+        ref_name = None
+
+    def bind(self, field_name, parent):
+        if self.method_name is None:
+            method_name = 'get_{field_name}_serializer'.format(field_name=field_name)
+            self.method_name = method_name
+
+        super().bind(field_name, parent)
+
+    @cached_property
+    def serializer(self) -> serializers.Serializer:
+        method = getattr(self.parent, self.method_name)
+        _serializer = method()
+        # 设置serializer的parent值，否则在serializer实例中获取parent会出现断层
+        setattr(_serializer, 'parent', self.parent)
+        return _serializer
+
+    @cached_property
+    def fields(self):
+        """
+        重写此方法因为在 BindingDict 中要设置每一个 field 的 parent 为 `serializer`,
+        这样在调用 field.parent 时, 才会达到预期的结果，
+        比如: serializers.SerializerMethodField
+        """
+        return self.serializer.fields
+
+    def run_validation(self, data=serializers.empty):
+        return self.serializer.run_validation(data)
+
+    def to_representation(self, instance):
+        return self.serializer.to_representation(instance)
+
+    def get_initial(self):
+        return self.serializer.get_initial()
+
+
+# Other Serializer
+# ----------------
 
 
 class EmptySerializer(Serializer):
@@ -23,3 +81,5 @@ class AdaptedBulkListSerializer(BulkListSerializerMixin, BulkListSerializer):
 
 class CeleryTaskSerializer(serializers.Serializer):
     task = serializers.CharField(read_only=True)
+
+

apps/common/fields/__init__.py

@@ -1,6 +1,4 @@
 # -*- coding: utf-8 -*-
 #
 
-from .form import *
 from .model import *
-from .serializer import *

apps/common/fields/form.py

@@ -1,62 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-import json
-
-from django import forms
-import six
-from django.core.exceptions import ValidationError
-from django.utils.translation import ugettext as _
-from ..utils import signer
-
-
-__all__ = [
-    'FormDictField', 'FormEncryptCharField', 'FormEncryptDictField',
-    'FormEncryptMixin',
-]
-
-
-class FormDictField(forms.Field):
-    widget = forms.Textarea
-
-    def to_python(self, value):
-        """Returns a Python boolean object."""
-        # Explicitly check for the string 'False', which is what a hidden field
-        # will submit for False. Also check for '0', since this is what
-        # RadioSelect will provide. Because bool("True") == bool('1') == True,
-        # we don't need to handle that explicitly.
-        if isinstance(value, six.string_types):
-            value = value.replace("'", '"')
-            try:
-                value = json.loads(value)
-                return value
-            except json.JSONDecodeError:
-                return ValidationError(_("Not a valid json"))
-        else:
-            return ValidationError(_("Not a string type"))
-
-    def validate(self, value):
-        if isinstance(value, ValidationError):
-            raise value
-        if not value and self.required:
-            raise ValidationError(self.error_messages['required'], code='required')
-
-    def has_changed(self, initial, data):
-        # Sometimes data or initial may be a string equivalent of a boolean
-        # so we should run it through to_python first to get a boolean value
-        return self.to_python(initial) != self.to_python(data)
-
-
-class FormEncryptMixin:
-    pass
-
-
-class FormEncryptCharField(FormEncryptMixin, forms.CharField):
-    pass
-
-
-class FormEncryptDictField(FormEncryptMixin, FormDictField):
-    pass
-
-
-
-

apps/common/fields/serializer.py

@@ -1,121 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-
-from rest_framework import serializers
-import six
-
-
-__all__ = [
-    'StringIDField', 'StringManyToManyField', 'ChoiceDisplayField',
-    'CustomMetaDictField'
-]
-
-
-class StringIDField(serializers.Field):
-    def to_representation(self, value):
-        return {"pk": value.pk, "name": value.__str__()}
-
-
-class StringManyToManyField(serializers.RelatedField):
-    def to_representation(self, value):
-        return value.__str__()
-
-
-class ChoiceDisplayField(serializers.ChoiceField):
-    def __init__(self, *args, **kwargs):
-        super(ChoiceDisplayField, self).__init__(*args, **kwargs)
-        self.choice_strings_to_display = {
-            six.text_type(key): value for key, value in self.choices.items()
-        }
-
-    def to_representation(self, value):
-        if value is None:
-            return value
-        return {
-            'value': self.choice_strings_to_values.get(six.text_type(value), value),
-            'display': self.choice_strings_to_display.get(six.text_type(value), value),
-        }
-
-
-class DictField(serializers.DictField):
-    def to_representation(self, value):
-        if not value or not isinstance(value, dict):
-            value = {}
-        return super().to_representation(value)
-
-
-class CustomMetaDictField(serializers.DictField):
-    """
-    In use:
-    RemoteApp params field
-    CommandStorage meta field
-    ReplayStorage meta field
-    """
-    type_fields_map = {}
-    default_type = None
-    convert_key_remove_type_prefix = False
-    convert_key_to_upper = False
-
-    def filter_attribute(self, attribute, instance):
-        fields = self.type_fields_map.get(instance.type, [])
-        for field in fields:
-            if field.get('write_only', False):
-                attribute.pop(field['name'], None)
-        return attribute
-
-    def get_attribute(self, instance):
-        """
-        序列化时调用
-        """
-        attribute = super().get_attribute(instance)
-        attribute = self.filter_attribute(attribute, instance)
-        return attribute
-
-    def convert_value_key_remove_type_prefix(self, dictionary, value):
-        if not self.convert_key_remove_type_prefix:
-            return value
-        tp = dictionary.get('type')
-        prefix = '{}_'.format(tp)
-        convert_value = {}
-        for k, v in value.items():
-            if k.lower().startswith(prefix):
-                k = k.lower().split(prefix, 1)[1]
-            convert_value[k] = v
-        return convert_value
-
-    def convert_value_key_to_upper(self, value):
-        if not self.convert_key_to_upper:
-            return value
-        convert_value = {k.upper(): v for k, v in value.items()}
-        return convert_value
-
-    def convert_value_key(self, dictionary, value):
-        value = self.convert_value_key_remove_type_prefix(dictionary, value)
-        value = self.convert_value_key_to_upper(value)
-        return value
-
-    def filter_value_key(self, dictionary, value):
-        tp = dictionary.get('type')
-        fields = self.type_fields_map.get(tp, [])
-        fields_names = [field['name'] for field in fields]
-        filter_value = {k: v for k, v in value.items() if k in fields_names}
-        return filter_value
-
-    @staticmethod
-    def strip_value(value):
-        new_value = {}
-        for k, v in value.items():
-            if isinstance(v, str):
-                v = v.strip()
-            new_value[k] = v
-        return new_value
-
-    def get_value(self, dictionary):
-        """
-        反序列化时调用
-        """
-        value = super().get_value(dictionary)
-        value = self.convert_value_key(dictionary, value)
-        value = self.filter_value_key(dictionary, value)
-        value = self.strip_value(value)
-        return value

apps/common/mixins/api.py

@@ -28,19 +28,39 @@ class JSONResponseMixin(object):
         return JsonResponse(context)
 
 
+# SerializerMixin
+# ----------------------
+
+
 class SerializerMixin:
-    def get_serializer_class(self):
+    """ 根据用户请求动作的不同，获取不同的 `serializer_class `"""
+
+    serializer_classes = None
+
+    def get_serializer_class_by_view_action(self):
+        if not hasattr(self, 'serializer_classes'):
+            return None
+        if not isinstance(self.serializer_classes, dict):
+            return None
+        action = self.request.query_params.get('action')
+
         serializer_class = None
-        if hasattr(self, 'serializer_classes') and isinstance(self.serializer_classes, dict):
-            if self.action in ['list', 'metadata'] and self.request.query_params.get('draw'):
-                serializer_class = self.serializer_classes.get('display')
-            if serializer_class is None:
-                serializer_class = self.serializer_classes.get(
-                    self.action, self.serializer_classes.get('default')
-                )
-        if serializer_class:
-            return serializer_class
-        return super().get_serializer_class()
+        if action:
+            # metadata方法 使用 action 参数获取
+            serializer_class = self.serializer_classes.get(action)
+        if serializer_class is None:
+            serializer_class = self.serializer_classes.get(self.action)
+        if serializer_class is None:
+            serializer_class = self.serializer_classes.get('display')
+        if serializer_class is None:
+            serializer_class = self.serializer_classes.get('default')
+        return serializer_class
+
+    def get_serializer_class(self):
+        serializer_class = self.get_serializer_class_by_view_action()
+        if serializer_class is None:
+            serializer_class = super().get_serializer_class()
+        return serializer_class
 
 
 class ExtraFilterFieldsMixin:

apps/common/mixins/serializers.py

@@ -124,6 +124,22 @@ class BulkListSerializerMixin(object):
 
         return ret
 
+    def create(self, validated_data):
+        ModelClass = self.child.Meta.model
+        use_model_bulk_create = getattr(self.child.Meta, 'use_model_bulk_create', False)
+        model_bulk_create_kwargs = getattr(self.child.Meta, 'model_bulk_create_kwargs', {})
+
+        if use_model_bulk_create:
+            to_create = [
+                ModelClass(**attrs) for attrs in validated_data
+            ]
+            objs = ModelClass._default_manager.bulk_create(
+                to_create, **model_bulk_create_kwargs
+            )
+            return objs
+        else:
+            return super().create(validated_data)
+
 
 class BaseDynamicFieldsPlugin:
     def __init__(self, serializer):

apps/common/serializers.py

@@ -1,6 +0,0 @@
-"""
-老的代码统一到 `apps/common/drf/serializers.py` 中，
-之后此文件废弃
-"""
-
-from common.drf.serializers import AdaptedBulkListSerializer, CeleryTaskSerializer

apps/common/tasks.py

@@ -4,7 +4,6 @@ from celery import shared_task
 
 from .utils import get_logger
 
-
 logger = get_logger(__file__)
 
 

apps/common/utils/common.py

@@ -1,6 +1,7 @@
 # -*- coding: utf-8 -*-
 #
 import re
+import data_tree
 from collections import OrderedDict
 from itertools import chain
 import logging
@@ -10,6 +11,8 @@ from functools import wraps
 import time
 import ipaddress
 import psutil
+from django.utils.translation import ugettext_lazy as _
+from ..exceptions import JMSException
 
 
 UUID_PATTERN = re.compile(r'\w{8}(-\w{4}){3}-\w{12}')

apps/common/utils/lock.py

@@ -1,4 +1,5 @@
 from functools import wraps
+import threading
 
 from redis_lock import Lock as RedisLock
 from redis import Redis
@@ -35,11 +36,16 @@ class DistributedLock(RedisLock):
         self._blocking = blocking
 
     def __enter__(self):
+        thread_id = threading.current_thread().ident
+        logger.debug(f'DISTRIBUTED_LOCK: <thread_id:{thread_id}> attempt to acquire <lock:{self._name}> ...')
         acquired = self.acquire(blocking=self._blocking)
         if self._blocking and not acquired:
+            logger.debug(f'DISTRIBUTED_LOCK: <thread_id:{thread_id}> was not acquired <lock:{self._name}>, but blocking=True')
             raise EnvironmentError("Lock wasn't acquired, but blocking=True")
         if not acquired:
+            logger.debug(f'DISTRIBUTED_LOCK: <thread_id:{thread_id}> acquire <lock:{self._name}> failed')
             raise AcquireFailed
+        logger.debug(f'DISTRIBUTED_LOCK: <thread_id:{thread_id}> acquire <lock:{self._name}> ok')
         return self
 
     def __exit__(self, exc_type=None, exc_value=None, traceback=None):

apps/jumpserver/conf.py

@@ -461,9 +461,6 @@ class DynamicConfig:
             backends.insert(0, 'authentication.backends.api.SSOAuthentication')
         return backends
 
-    def AUTH_DB(self):
-        return len(self.AUTHENTICATION_BACKENDS()) == 2
-
     def XPACK_LICENSE_IS_VALID(self):
         if not HAS_XPACK:
             return False

apps/jumpserver/context_processor.py

@@ -13,6 +13,7 @@ def jumpserver_processor(request):
         'LOGO_TEXT_URL': static('img/logo_text.png'),
         'LOGIN_IMAGE_URL': static('img/login_image.png'),
         'FAVICON_URL': static('img/facio.ico'),
+        'LOGIN_CAS_LOGO_URL': static('img/login_cas_logo.png'),
         'JMS_TITLE': 'JumpServer',
         'VERSION': settings.VERSION,
         'COPYRIGHT': 'FIT2CLOUD 飞致云' + ' © 2014-2020',

apps/jumpserver/settings/auth.py

@@ -9,9 +9,6 @@ from ..const import CONFIG, DYNAMIC, PROJECT_DIR
 OTP_ISSUER_NAME = CONFIG.OTP_ISSUER_NAME
 OTP_VALID_WINDOW = CONFIG.OTP_VALID_WINDOW
 
-# Auth DB
-AUTH_DB = DYNAMIC.AUTH_DB
-
 # Auth LDAP settings
 AUTH_LDAP = DYNAMIC.AUTH_LDAP
 AUTH_LDAP_SERVER_URI = DYNAMIC.AUTH_LDAP_SERVER_URI