Merge 6e7c537956 into f4baa4c6d8

Fix callback output to align header with roles, tasks and play RECAP

SUMMARY
All three callback plugins (timer, roles, tasks) do not produce headers in the output that align with how we do "PLAY RECAP *********" that is generated by the "default" output callback. So these changes are provided to align the callback plugins and have a clean output that is much more readable.
Note, the asterisks are generated just as how they are for "PLAY RECAP", where it extends to the end of the terminal window.
I am considering this as a "bug" in that the output was never aligned, instead of a feature since this does not provide any new feature per-se.
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ansible.posix.timer
ansible.posix.profile_tasks
ansible.posix.profile_roles
ADDITIONAL INFORMATION


The new output is shown below. You can see the headers are now all the same and broken down by PLAY, ROLE, TASKS, and PLAYBOOK RECAP.

PLAY RECAP ***************************************************************************************************************************************************************************************
localhost                  : ok=14   changed=2    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   


ROLE RECAP ***************************************************************************************************************************************************************************************
awx.awx.subscriptions --------------------------------------------------- 4.81s
users ------------------------------------------------------------------- 2.89s
labels ------------------------------------------------------------------ 2.86s
settings ---------------------------------------------------------------- 2.46s
credential_types -------------------------------------------------------- 2.27s
organizations ----------------------------------------------------------- 2.21s
teams ------------------------------------------------------------------- 2.21s
awx.awx.license --------------------------------------------------------- 1.49s
dispatch ---------------------------------------------------------------- 0.58s
credentials ------------------------------------------------------------- 0.03s
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
total ------------------------------------------------------------------ 21.80s

TASKS RECAP **************************************************************************************************************************************************************************************
Get subscriptions with a filter ----------------------------------------------------------------------------------------------------------------------------------------------------------- 4.81s
redhat_cop.controller_configuration.users : Configure Users | Wait for finish the Users creation ------------------------------------------------------------------------------------------ 1.93s
redhat_cop.controller_configuration.labels : Configure Labels | Wait for finish the Label creation ---------------------------------------------------------------------------------------- 1.91s
redhat_cop.controller_configuration.settings : Configure Settings | Wait for finish the Settings creation --------------------------------------------------------------------------------- 1.75s
redhat_cop.controller_configuration.teams : Configure Teams | Wait for finish the Teams creation ------------------------------------------------------------------------------------------ 1.63s
redhat_cop.controller_configuration.organizations : Configure Controller Organizations | Wait for finish the organization creation -------------------------------------------------------- 1.62s
Attach to a pool -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.49s
redhat_cop.controller_configuration.credential_types : Add Credential Types --------------------------------------------------------------------------------------------------------------- 1.32s
redhat_cop.controller_configuration.users : Add controller user --------------------------------------------------------------------------------------------------------------------------- 0.96s
redhat_cop.controller_configuration.labels : Add a label to Controller -------------------------------------------------------------------------------------------------------------------- 0.95s
redhat_cop.controller_configuration.credential_types : Configure Controller Credential Types | Wait for finish the credential types creation ---------------------------------------------- 0.94s
redhat_cop.controller_configuration.settings : Update Ansible Controller Settings from dictionary or list of dictionaries ----------------------------------------------------------------- 0.70s
redhat_cop.controller_configuration.organizations : Add organizations --------------------------------------------------------------------------------------------------------------------- 0.59s
redhat_cop.controller_configuration.teams : Create Ansible Controller Team ---------------------------------------------------------------------------------------------------------------- 0.58s
Run redhat_cop.controller_configuration.{{ __role.role }} role ---------------------------------------------------------------------------------------------------------------------------- 0.58s
redhat_cop.controller_configuration.credentials : Add Credentials ------------------------------------------------------------------------------------------------------------------------- 0.03s

PLAYBOOK RECAP ***********************************************************************************************************************************************************************************
Playbook run took 0 days, 0 hours, 0 minutes, 21 seconds


Current output (how it is today) for comparison:
PLAY RECAP ***************************************************************************************************************************************************************************************
localhost                  : ok=14   changed=2    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

Playbook run took 0 days, 0 hours, 0 minutes, 23 seconds
Tuesday 20 September 2022  15:00:49 -0700 (0:00:00.038)       0:00:23.039 ***** 
=============================================================================== 
Get subscriptions with a filter ----------------------------------------------------------------------------------------------------------------------------------------------------------- 5.49s
redhat_cop.controller_configuration.labels : Configure Labels | Wait for finish the Label creation ---------------------------------------------------------------------------------------- 1.98s
redhat_cop.controller_configuration.users : Configure Users | Wait for finish the Users creation ------------------------------------------------------------------------------------------ 1.92s
redhat_cop.controller_configuration.organizations : Configure Controller Organizations | Wait for finish the organization creation -------------------------------------------------------- 1.84s
redhat_cop.controller_configuration.settings : Configure Settings | Wait for finish the Settings creation --------------------------------------------------------------------------------- 1.78s
redhat_cop.controller_configuration.teams : Configure Teams | Wait for finish the Teams creation ------------------------------------------------------------------------------------------ 1.61s
Attach to a pool -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.53s
redhat_cop.controller_configuration.credential_types : Add Credential Types --------------------------------------------------------------------------------------------------------------- 1.35s
redhat_cop.controller_configuration.users : Add controller user --------------------------------------------------------------------------------------------------------------------------- 0.95s
redhat_cop.controller_configuration.labels : Add a label to Controller -------------------------------------------------------------------------------------------------------------------- 0.95s
redhat_cop.controller_configuration.credential_types : Configure Controller Credential Types | Wait for finish the credential types creation ---------------------------------------------- 0.94s
redhat_cop.controller_configuration.settings : Update Ansible Controller Settings from dictionary or list of dictionaries ----------------------------------------------------------------- 0.73s
Run redhat_cop.controller_configuration.{{ __role.role }} role ---------------------------------------------------------------------------------------------------------------------------- 0.63s
redhat_cop.controller_configuration.organizations : Add organizations --------------------------------------------------------------------------------------------------------------------- 0.61s
redhat_cop.controller_configuration.teams : Create Ansible Controller Team ---------------------------------------------------------------------------------------------------------------- 0.60s
redhat_cop.controller_configuration.credentials : Add Credentials ------------------------------------------------------------------------------------------------------------------------- 0.04s
Tuesday 20 September 2022  15:00:49 -0700 (0:00:00.043)       0:00:23.041 ***** 
=============================================================================== 
awx.awx.subscriptions --------------------------------------------------- 5.49s
labels ------------------------------------------------------------------ 2.93s
users ------------------------------------------------------------------- 2.88s
settings ---------------------------------------------------------------- 2.51s
organizations ----------------------------------------------------------- 2.45s
credential_types -------------------------------------------------------- 2.29s
teams ------------------------------------------------------------------- 2.21s
awx.awx.license --------------------------------------------------------- 1.53s
dispatch ---------------------------------------------------------------- 0.63s
credentials ------------------------------------------------------------- 0.04s
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
total ------------------------------------------------------------------ 22.95s

Reviewed-by: Hideki Saito <saito@fgrep.org>

changelogs/fragments/387_callback_output_header.yml

@@ -0,0 +1,3 @@
+---
+minor_changes:
+  - callback plugins - Add recap information to timer, profile_roles and profile_tasks callback outputs (https://github.com/ansible-collections/ansible.posix/pull/387).

changelogs/fragments/568_update_authorized_key.yml

@@ -0,0 +1,3 @@
+---
+minor_changes:
+  - authorized_keys - allow using absolute path to a file as a SSH key(s) source (https://github.com/ansible-collections/ansible.posix/pull/568)

plugins/action/synchronize.py

@@ -339,6 +339,8 @@ class ActionModule(ActionBase):
         dest = _tmp_args.get('dest', None)
         if src is None or dest is None:
             return dict(failed=True, msg="synchronize requires both src and dest parameters are set")
+        if isinstance(src, str):
+            src = [src]
 
         # Determine if we need a user@ and a password
         user = None
@@ -365,11 +367,11 @@ class ActionModule(ActionBase):
             # use the mode to define src and dest's url
             if _tmp_args.get('mode', 'push') == 'pull':
                 # src is a remote path: <user>@<host>, dest is a local path
-                src = self._process_remote(_tmp_args, src_host, src, user, inv_port in localhost_ports)
+                src = [self._process_remote(_tmp_args, src_host, e, user, inv_port in localhost_ports) for e in src]
                 dest = self._process_origin(dest_host, dest, user)
             else:
                 # src is a local path, dest is a remote path: <user>@<host>
-                src = self._process_origin(src_host, src, user)
+                src = [self._process_origin(src_host, e, user) for e in src]
                 dest = self._process_remote(_tmp_args, dest_host, dest, user, inv_port in localhost_ports)
 
             password = dest_host_inventory_vars.get('ansible_ssh_pass', None) or dest_host_inventory_vars.get('ansible_password', None)
@@ -378,7 +380,7 @@ class ActionModule(ActionBase):
         else:
             # Still need to munge paths (to account for roles) even if we aren't
             # copying files between hosts
-            src = self._get_absolute_path(path=src)
+            src = [self._get_absolute_path(path=e) for e in src]
             dest = self._get_absolute_path(path=dest)
 
         _tmp_args['_local_rsync_password'] = password

plugins/callback/profile_roles.py

@@ -128,7 +128,10 @@ class CallbackModule(CallbackBase):
         self._display_tasktime()
 
     def playbook_on_stats(self, stats):
-        self._display_tasktime()
+        # Align summary report header with other callback plugin summary
+        self._display.banner("ROLES RECAP")
+
+        self._display.display(tasktime())
         self._display.display(filled("", fchar="="))
 
         timestamp(self)

plugins/callback/profile_tasks.py

@@ -193,7 +193,10 @@ class CallbackModule(CallbackBase):
         self._display_tasktime()
 
     def playbook_on_stats(self, stats):
-        self._display_tasktime()
+        # Align summary report header with other callback plugin summary
+        self._display.banner("TASKS RECAP")
+
+        self._display.display(tasktime())
         self._display.display(filled("", fchar="="))
 
         timestamp(self)

plugins/callback/timer.py

@@ -46,4 +46,6 @@ class CallbackModule(CallbackBase):
     def v2_playbook_on_stats(self, stats):
         end_time = datetime.utcnow()
         runtime = end_time - self.start_time
-        self._display.display("Playbook run took %s days, %s hours, %s minutes, %s seconds" % (self.days_hours_minutes_seconds(runtime)))
+        # Align summary report header with other callback plugin summary
+        self._display.banner("PLAYBOOK RECAP")
+        self._display.display("Playbook run took %s days, %s hours, %s minutes, %s seconds\n\r" % (self.days_hours_minutes_seconds(runtime)))

plugins/modules/authorized_key.py

@@ -24,6 +24,7 @@ options:
   key:
     description:
       - The SSH public key(s), as a string or (since Ansible 1.9) url (https://github.com/username.keys).
+      - You can also use V(file://) prefix to search remote for a file with SSH key(s).
     type: str
     required: true
   path:
@@ -96,6 +97,12 @@ EXAMPLES = r'''
     state: present
     key: https://github.com/charlie.keys
 
+- name: Set authorized keys taken from path on controller node
+  ansible.posix.authorized_key:
+    user: charlie
+    state: present
+    key: file:///home/charlie/.ssh/id_rsa.pub
+
 - name: Set authorized keys taken from url using lookup
   ansible.posix.authorized_key:
     user: charlie
@@ -223,6 +230,7 @@ from operator import itemgetter
 from ansible.module_utils._text import to_native
 from ansible.module_utils.basic import AnsibleModule
 from ansible.module_utils.urls import fetch_url
+from ansible.module_utils.six.moves.urllib.parse import urlparse
 
 
 class keydict(dict):
@@ -556,7 +564,7 @@ def enforce_state(module, params):
     follow = params.get('follow', False)
     error_msg = "Error getting key from: %s"
 
-    # if the key is a url, request it and use it as key source
+    # if the key is a url or file, request it and use it as key source
     if key.startswith("http"):
         try:
             resp, info = fetch_url(module, key)
@@ -570,6 +578,19 @@ def enforce_state(module, params):
         # resp.read gives bytes on python3, convert to native string type
         key = to_native(key, errors='surrogate_or_strict')
 
+    if key.startswith("file"):
+        # if the key is an absolute path, check for existense and use it as a key source
+        key_path = urlparse(key).path
+        if not os.path.exists(key_path):
+            module.fail_json(msg="Path to a key file not found: %s" % key_path)
+        if not os.path.isfile(key_path):
+            module.fail_json(msg="Path to a key is a directory and must be a file: %s" % key_path)
+        try:
+            with open(key_path, 'r') as source_fh:
+                key = source_fh.read()
+        except OSError as e:
+            module.fail_json(msg="Failed to read key file %s : %s" % (key_path, to_native(e)))
+
     # extract individual keys into an array, skipping blank lines and comments
     new_keys = [s for s in key.splitlines() if s and not s.startswith('#')]
 

plugins/modules/synchronize.py

@@ -361,6 +361,17 @@ EXAMPLES = r'''
         src: /tmp/localpath/
         dest: /tmp/remotepath
         rsync_path: /usr/gnu/bin/rsync
+
+# Source files from multiple folders and merge them on the remote
+# Files of the same name in /tmp/path_c/ will take precedence over those in /tmp/path_b/, and same for path_b to path_a
+- name: Copy files from multiple folders and merge them into dest
+  ansible.posix.synchronize:
+    src: 
+    - /tmp/path_a/
+    - /tmp/path_b/
+    - /tmp/path_c/
+    dest: /tmp/dest/
+    recursive: True
 '''
 
 
@@ -396,9 +407,9 @@ def substitute_controller(path):
 
 
 def is_rsh_needed(source, dest):
-    if source.startswith('rsync://') or dest.startswith('rsync://'):
+    if all(e.startswith('rsync://') for e in source) or dest.startswith('rsync://'):
         return False
-    if ':' in source or ':' in dest:
+    if any(':' in e for e in source) or ':' in dest:
         return True
     return False
 
@@ -406,7 +417,7 @@ def is_rsh_needed(source, dest):
 def main():
     module = AnsibleModule(
         argument_spec=dict(
-            src=dict(type='path', required=True),
+            src=dict(type='list', required=True),
             dest=dict(type='path', required=True),
             dest_port=dict(type='int'),
             delete=dict(type='bool', default=False),
@@ -540,11 +551,10 @@ def main():
     if dirs:
         cmd.append('--dirs')
 
-    if source.startswith('rsync://') and dest.startswith('rsync://'):
+    if all(e.startswith('rsync://') for e in source) and dest.startswith('rsync://'):
         module.fail_json(msg='either src or dest must be a localhost', rc=1)
 
     if is_rsh_needed(source, dest):
-
         # https://github.com/ansible/ansible/issues/15907
         has_rsh = False
         for rsync_opt in rsync_opts:
@@ -600,7 +610,7 @@ def main():
     changed_marker = '<<CHANGED>>'
     cmd.append('--out-format=%s' % shlex_quote(changed_marker + '%i %n%L'))
 
-    cmd.append(shlex_quote(source))
+    [cmd.append(shlex_quote(e)) for e in source]
     cmd.append(shlex_quote(dest))
     cmdstr = ' '.join(cmd)
 

tests/integration/targets/authorized_key/defaults/main.yml

@@ -35,3 +35,5 @@ multiple_keys_comments: |
   ssh-rsa DATA_BASIC 1@testing
   # I like adding comments yo-dude-this-is-not-a-key INVALID_DATA 2@testing
   ecdsa-sha2-nistp521 ECDSA_DATA 4@testing
+
+key_path: /tmp/id_rsa.pub

tests/integration/targets/authorized_key/tasks/check_path.yml

@@ -0,0 +1,32 @@
+---
+- name: Create key file for test
+  ansible.builtin.copy:
+    dest: "{{ key_path }}"
+    content: "{{ rsa_key_basic }}"
+    mode: "0600"
+
+- name: Add key using path
+  ansible.posix.authorized_key:
+    user: root
+    key: file://{{ key_path }}
+    state: present
+    path: "{{ output_dir | expanduser }}/authorized_keys"
+  register: result
+
+- name: Assert that the key was added
+  ansible.builtin.assert:
+    that:
+      - result.changed == true
+
+- name: Add key using path again
+  ansible.posix.authorized_key:
+    user: root
+    key: file://{{ key_path }}
+    state: present
+    path: "{{ output_dir | expanduser }}/authorized_keys"
+  register: result
+
+- name: Assert that no changes were applied
+  ansible.builtin.assert:
+    that:
+      - result.changed == false

tests/integration/targets/authorized_key/tasks/main.yml

@@ -31,3 +31,6 @@
 
 - name: Test for the management of comments with key
   ansible.builtin.import_tasks: comments.yml
+
+- name: Test for specifying key as a path
+  ansible.builtin.import_tasks: check_path.yml