v3.0.0-rc1 (#9322)

* perf：automation

* pref: 修改账号推送

* perf: 修改 assets

* perf: 修改 accounts

* feat: 优化代码

* fix: 修复 ObjectRelatedField 获取 value attr 时先判断是否有 attr 属性

* perf: 增加翻译

* feat: 增加部分翻译

* feat: 去除无用列

* perf: ticket remove app

* fix: 修复创建账号备份任务失败的问题

* perf: 添加 accounts app

* perf: ticket type serializer (#9252)

Co-authored-by: feng <1304903146@qq.com>

* perf: ticket

* perf: 修改 accounts api

* perf: 优化 AssetPermissionSerializer fields 顺序

* perf: 修改 accounts

* feat: 限制常用用户名api返回长度

* feat: 限制常用用户名api返回长度

* perf: 修改 LoginAssetACL 序列类，增加 users_username_group, accounts_username_group... 字段

* perf: 修改 CommandFilterACLSerializer 增加 command_groups_amount 字段

* perf: 修改rbac API啥的 (#9254)

* perf: migrate

* perf: 修改 AssetPermedSerializer domain 字段类型

* perf: 放开push account 权限位

* perf: 修改 accounts

* perf: 修改 LoginACLSerializer 字段类型

* pref: 修改数据库 migrations

* perf: filter asset systemuser

* perf: 修改 SessionSerializer 字段类型

* pref: 修改 applet host

* perf: 修改 SessionCommandSerializer 字段类型

* perf: 修改 accounts import

* perf: 修改 celery datetime

* perf: 修改 asset serializer

* pref: 修改 labeled field

* feat: 修改翻译

* perf: 修改 JobSerializer 字段类型

* feat: 支持使用 ws 发送终断任务

* perf: add AccessTokenAuthentication

* perf: 修改 BaseStorageSerializer 字段类型

* perf: 修改 AppletHostSerializer 字段类型

* perf: signal event

* perf: asset types automations (#9259)

Co-authored-by: feng <1304903146@qq.com>

* perf: 修改下载 rdp 文件时返回的 address 地址信息为空的问题

* perf: 修改 AssetSerializer.accounts.secret 为 write_only; 修改 DomainWithGatewaySerializer.gateways 返回 account 信息及 secret 字段;

* perf: automation 干库 (#9260)

Co-authored-by: feng <1304903146@qq.com>

* perf: account push api

* feat: 修改迁移文件

* feat: 删除无用代码

* feat: 优化部分资源无操作日志

* perf: 修改 account

* perf: perm tree

* perf: asset serializers retrieve

* perf: 格式化代码

* perf: AutomationExecution (#9268)

Co-authored-by: feng <1304903146@qq.com>

* perf: AssetDetailSerializer 和 Asset Model 添加 specific_info 字段;

* perf: 修改账号推送

* feat: handle ws heartbeat status

* perf: k8s tree (#9269)

Co-authored-by: feng <1304903146@qq.com>

* perf: 修改账号推送

* perf: 修改 asset detail serializer

* fix: 修复 windows 不能运行 powershell 命令的问题

* feat: 支持按照资源时间线查看操作活动

* feat: 翻译

* feat: 优化操作日志

* perf: asset clone

* fix: 错误的修改改回去

* perf: create asset account

* feat: 增加task 刷新续传功能

* fix: applet host deloypment filter host

* perf: 修改了 common 结构，和 push accounts

* perf: 整理 common 结构

* perf: 修改 const import

* perf: 修改 allow bulk destroy

* fix: applet host search fileds

* perf: applet bulk delete

* fix: applet list 404

* perf: 修改 common view

* feat: 增加一些翻译, 修复 playbook 上传的错误

* fix: 修改错别字

* perf: 修改 applets status

* perf: 修改网关 api

* perf: automateion (#9281)

Co-authored-by: feng <1304903146@qq.com>
Co-authored-by: feng626 <57284900+feng626@users.noreply.github.com>

* perf: 失效 connect methods 当 applet 删除 或者 host 删除

* perf: 网关账号的密码类型改成 LabelField

* perf: chrome applet script

* perf: verify code ttl (#9282)

Co-authored-by: feng <1304903146@qq.com>

* perf: database ping

* perf: ws

* perf: 修改网关创建

* perf: account task org (#9285)

Co-authored-by: feng <1304903146@qq.com>

* perf: asset test api

* perf: port 添加 account

* pref: 修改 db mapper permission

* fix: db port mapper list api

* perf: account change secret (#9286)

Co-authored-by: feng <1304903146@qq.com>

* perf: 修改 setup_eager_loading

* perf: SecretStrategy

* feat: 修改 ConnectionToken Create API 支持校验 ACL 逻辑

* feat: 修改 ConnectionToken Create API 支持校验 ACL 逻辑

* feat: 修改 ConnectionToken Create API 支持校验 ACL 逻辑

* pref: web database 信号转发

* perf: account push automation

* perf: push filter account

* perf: 修改 publish 版本

* perf: 修改网关

* fix: 修改资产 Specific 信息中 JSONField 字段返回 json.loads 对象

* feat: 远程应用内置Navicat Premium 16

* feat: 更新下载链接

* feat: 整理代码格式

* perf: 修改 terminal point

* perf: update chrome applet script

* fix: 资产 specific 获取 JSONField 时, 判断值的类型不为 list, dict

* perf: domain (#9292)

Co-authored-by: feng <1304903146@qq.com>

* perf: 优化 endpoint 监听端口，仅 oracle 动态

* perf: 修改翻译

* perf: 修改文案

* perf: 修改缺失的翻译

* perf: 修改 endpoint help text

* feat: 还原格式

* feat: 去掉基类

* feat: 增加特权账号字段

* perf: decode content

* fix: check pid

* perf: 修改 smart endpoint

* perf: 修改 endpoint mysql default port

* feat: 优化

* perf: 修改 endpoint mysql default port

* perf: gateway test (#9295)

Co-authored-by: feng <1304903146@qq.com>

* perf: migrate

* perf: 修改 endpoint mysql default port

* fix: 修复获取任务执行结果死循环

* feat: 作业审计日志增加字段

* fix: add on_transaction_commit task post save

* perf: gateway (#9297)

Co-authored-by: feng <1304903146@qq.com>

* feat: 过滤 jumpserver 自动产生的用户

* fix: 修复ops节点选择的问题

* fix: 修改 统一 connection-token 和 command 的 review API 返回数据 from_ticket_info

* perf: change secret (#9298)

Co-authored-by: feng <1304903146@qq.com>

* perf: 修改 db port manager

* perf: 修改 db port manager

* perf: add celery log mark

* perf: remove debug log data

* fix: navicat use manual type

* fix: remove navicate download url

* perf: push_account_enabled (#9301)

Co-authored-by: feng <1304903146@qq.com>

* fix: 修改navicat启动程序MD5值

* perf: push account (#9303)

Co-authored-by: feng <1304903146@qq.com>

* feat: Redis/MongoDB 支持SSL

* fix: 修改授权规则过滤字段 node_name,node_id; 修复获取授权节点下的资产为空的问题;

* perf: push account button (#9305)

Co-authored-by: feng <1304903146@qq.com>

* perf: account push

* fix: 修复获取 /user//assets/tree/ 返回用户授权的所有资产

* perf: asset ping (#9307)

Co-authored-by: feng <1304903146@qq.com>

* perf: asset enabled_info

* perf: 优化activity记录都保存至operatelog中

* feat: 远程应用navicat支持试用版连接

* perf: 优化迁移文件

* perf: 修改资产列表 API category type 字段 choices 根据 category 进行返回

* fix

* perf: 修改账号列表 API 解决根据 node_id asset_id 搜索账号列表无效的问题

* fix: navicat dba账号登录

* perf: 优化navicat连接

* perf: 修改账号列表 Model Manager 继承自 OrgManager，解决组织过滤问题

* perf: 修改账号列表 Filter 支持根据 platform,category,type 字段搜索

* perf: change secret email (#9312)

Co-authored-by: feng <1304903146@qq.com>

* feat: 保证认证信息一定清理

* perf: add mariadb

* perf: 修改资产类型树数量统计资产或账号

* perf: applet chrome quit

* perf: 优化关闭欢迎页面

* fix

* perf: executed amount

* perf: 修改 built-in applet installation

* perf: 修改资产列表增加标签搜索

* perf: 修改资产列表增加标签搜索

* perf: account task automation (#9319)

Co-authored-by: feng <1304903146@qq.com>

* perf: account trigger

* perf: 修改系统设置文案：批量命令执行 -> 作业中心

* perf: 优化migrate (#9320)

Co-authored-by: feng <1304903146@qq.com>

* perf: 修改资产节点树 API，支持搜索资产、节点

* perf: audit dashboard (#9321)

Co-authored-by: feng <1304903146@qq.com>

* fix: 修改 has_perm 权限判断兼容 list 和 str 类型

* perf: 修改一些换行

* perf: 修改 ansible config

* fix: oracle依赖文件地址错误 (#9324)

* perf: ansible mudules

* perf: 修改 runner host cwd

Co-authored-by: ibuler <ibuler@qq.com>
Co-authored-by: Aaron3S <chenyang@fit2cloud.com>
Co-authored-by: Bai <baijiangjie@gmail.com>
Co-authored-by: feng <1304903146@qq.com>
Co-authored-by: feng626 <57284900+feng626@users.noreply.github.com>
Co-authored-by: Eric <xplzv@126.com>
Co-authored-by: jiangweidong <weidong.jiang@fit2cloud.com>
Co-authored-by: jiangweidong <80373698+Hi-JWD@users.noreply.github.com>

apps/accounts/migrations/0003_automation.py

@@ -0,0 +1,194 @@
+# Generated by Django 3.2.16 on 2022-12-30 08:08
+
+import common.db.encoder
+import common.db.fields
+from django.conf import settings
+from django.db import migrations, models
+import django.db.models.deletion
+import uuid
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ('assets', '0107_automation'),
+        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
+        ('accounts', '0002_auto_20220616_0021'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='AccountBaseAutomation',
+            fields=[
+            ],
+            options={
+                'verbose_name': 'Account automation task',
+                'proxy': True,
+                'indexes': [],
+                'constraints': [],
+            },
+            bases=('assets.baseautomation',),
+        ),
+        migrations.CreateModel(
+            name='AutomationExecution',
+            fields=[
+            ],
+            options={
+                'verbose_name': 'Automation execution',
+                'verbose_name_plural': 'Automation executions',
+                'permissions': [('view_changesecretexecution', 'Can view change secret execution'),
+                                ('add_changesecretexection', 'Can add change secret execution'),
+                                ('view_gatheraccountsexecution', 'Can view gather accounts execution'),
+                                ('add_gatheraccountsexecution', 'Can add gather accounts execution')],
+                'proxy': True,
+                'indexes': [],
+                'constraints': [],
+            },
+            bases=('assets.automationexecution',),
+        ),
+        migrations.CreateModel(
+            name='PushAccountAutomation',
+            fields=[
+                ('baseautomation_ptr',
+                 models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True,
+                                      primary_key=True, serialize=False, to='assets.baseautomation')),
+                ('secret_type', models.CharField(
+                    choices=[('password', 'Password'), ('ssh_key', 'SSH key'), ('access_key', 'Access key'),
+                             ('token', 'Token')], default='password', max_length=16, verbose_name='Secret type')),
+                ('secret_strategy', models.CharField(choices=[('specific', 'Specific password'),
+                                                              ('random_one', 'All assets use the same random password'),
+                                                              ('random_all',
+                                                               'All assets use different random password')],
+                                                     default='specific', max_length=16,
+                                                     verbose_name='Secret strategy')),
+                ('secret', common.db.fields.EncryptTextField(blank=True, null=True, verbose_name='Secret')),
+                ('password_rules', models.JSONField(default=dict, verbose_name='Password rules')),
+                ('ssh_key_change_strategy', models.CharField(
+                    choices=[('add', 'Append SSH KEY'), ('set', 'Empty and append SSH KEY'),
+                             ('set_jms', 'Replace (The key generated by JumpServer) ')], default='add', max_length=16,
+                    verbose_name='SSH key change strategy')),
+                ('triggers', models.JSONField(default=list, max_length=16, verbose_name='Triggers')),
+                ('username', models.CharField(max_length=128, verbose_name='Username')),
+                ('action', models.CharField(max_length=16, verbose_name='Action')),
+            ],
+            options={
+                'verbose_name': 'Push asset account',
+            },
+            bases=('accounts.accountbaseautomation', models.Model),
+        ),
+
+        migrations.CreateModel(
+            name='GatherAccountsAutomation',
+            fields=[
+                ('baseautomation_ptr',
+                 models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True,
+                                      primary_key=True, serialize=False, to='assets.baseautomation')),
+            ],
+            options={
+                'verbose_name': 'Gather asset accounts',
+            },
+            bases=('accounts.accountbaseautomation',),
+        ),
+        migrations.CreateModel(
+            name='VerifyAccountAutomation',
+            fields=[
+                ('baseautomation_ptr',
+                 models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True,
+                                      primary_key=True, serialize=False, to='assets.baseautomation')),
+            ],
+            options={
+                'verbose_name': 'Verify asset account',
+            },
+            bases=('accounts.accountbaseautomation',),
+        ),
+        migrations.CreateModel(
+            name='ChangeSecretRecord',
+            fields=[
+                ('created_by', models.CharField(blank=True, max_length=128, null=True, verbose_name='Created by')),
+                ('updated_by', models.CharField(blank=True, max_length=128, null=True, verbose_name='Updated by')),
+                ('date_created', models.DateTimeField(auto_now_add=True, null=True, verbose_name='Date created')),
+                ('date_updated', models.DateTimeField(auto_now=True, verbose_name='Date updated')),
+                ('comment', models.TextField(blank=True, default='', verbose_name='Comment')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('old_secret', common.db.fields.EncryptTextField(blank=True, null=True, verbose_name='Old secret')),
+                ('new_secret', common.db.fields.EncryptTextField(blank=True, null=True, verbose_name='Secret')),
+                ('date_started', models.DateTimeField(blank=True, null=True, verbose_name='Date started')),
+                ('date_finished', models.DateTimeField(blank=True, null=True, verbose_name='Date finished')),
+                ('status', models.CharField(default='pending', max_length=16)),
+                ('error', models.TextField(blank=True, null=True, verbose_name='Error')),
+                ('account',
+                 models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='accounts.account')),
+                ('asset', models.ForeignKey(null=True, on_delete=django.db.models.deletion.CASCADE, to='assets.asset')),
+                ('execution',
+                 models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='accounts.automationexecution')),
+            ],
+            options={
+                'verbose_name': 'Change secret record',
+            },
+        ),
+        migrations.CreateModel(
+            name='AccountBackupExecution',
+            fields=[
+                ('org_id',
+                 models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
+                ('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
+                ('date_start', models.DateTimeField(auto_now_add=True, verbose_name='Date start')),
+                ('timedelta', models.FloatField(default=0.0, null=True, verbose_name='Time')),
+                ('plan_snapshot',
+                 models.JSONField(blank=True, default=dict, encoder=common.db.encoder.ModelJSONFieldEncoder, null=True,
+                                  verbose_name='Account backup snapshot')),
+                ('trigger', models.CharField(choices=[('manual', 'Manual trigger'), ('timing', 'Timing trigger')],
+                                             default='manual', max_length=128, verbose_name='Trigger mode')),
+                ('reason', models.CharField(blank=True, max_length=1024, null=True, verbose_name='Reason')),
+                ('is_success', models.BooleanField(default=False, verbose_name='Is success')),
+                ('plan', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='execution',
+                                           to='accounts.accountbackupautomation', verbose_name='Account backup plan')),
+            ],
+            options={
+                'verbose_name': 'Account backup execution',
+                'ordering': ('-date_start',),
+            },
+        ),
+        migrations.CreateModel(
+            name='ChangeSecretAutomation',
+            fields=[
+                ('baseautomation_ptr',
+                 models.OneToOneField(auto_created=True, on_delete=django.db.models.deletion.CASCADE, parent_link=True,
+                                      primary_key=True, serialize=False, to='assets.baseautomation')),
+                ('secret_type', models.CharField(
+                    choices=[('password', 'Password'), ('ssh_key', 'SSH key'), ('access_key', 'Access key'),
+                             ('token', 'Token')], default='password', max_length=16, verbose_name='Secret type')),
+                ('secret_strategy', models.CharField(choices=[('specific', 'Specific password'),
+                                                              ('random_one', 'All assets use the same random password'),
+                                                              ('random_all',
+                                                               'All assets use different random password')],
+                                                     default='specific', max_length=16,
+                                                     verbose_name='Secret strategy')),
+                ('secret', common.db.fields.EncryptTextField(blank=True, null=True, verbose_name='Secret')),
+                ('password_rules', models.JSONField(default=dict, verbose_name='Password rules')),
+                ('ssh_key_change_strategy', models.CharField(
+                    choices=[('add', 'Append SSH KEY'), ('set', 'Empty and append SSH KEY'),
+                             ('set_jms', 'Replace (The key generated by JumpServer) ')], default='add', max_length=16,
+                    verbose_name='SSH key change strategy')),
+                ('recipients',
+                 models.ManyToManyField(blank=True, to=settings.AUTH_USER_MODEL, verbose_name='Recipient')),
+            ],
+            options={
+                'verbose_name': 'Change secret automation',
+            },
+            bases=('accounts.accountbaseautomation', models.Model),
+        ),
+        migrations.AlterModelOptions(
+            name='automationexecution',
+            options={'permissions': [('view_changesecretexecution', 'Can view change secret execution'),
+                                     ('add_changesecretexection', 'Can add change secret execution'),
+                                     ('view_gatheraccountsexecution', 'Can view gather accounts execution'),
+                                     ('add_gatheraccountsexecution', 'Can add gather accounts execution'),
+                                     ('view_pushaccountexecution', 'Can view push account execution'),
+                                     ('add_pushaccountexecution', 'Can add push account execution')],
+                     'verbose_name': 'Automation execution', 'verbose_name_plural': 'Automation executions'},
+        ),
+        migrations.AlterModelOptions(
+            name='changesecretrecord',
+            options={'ordering': ('-date_started',), 'verbose_name': 'Change secret record'},
+        ),
+    ]