SmartForward - itemid in ComposeMail object is changed

liverpoolfcfan

I have had a strange report on forwarding from a shared folder not working correctly. In investigating, it seems that the parsing of the ComposeMain:Source is behind the issue.

The original itemid passed from the Client shows a long itemid - which in zimbra’s case is a combined folderid:item - by the time the object gets passed to the backend and logged at the start of SendMail, the “folder:” is gone and just the original “item” part is left.

In the example below, the original
11/09/2020 10:03:09 [15627] [WBXML] I ComposeMail:ItemId
11/09/2020 10:03:09 [15627] [WBXML] I 62635aaf-db95-42a1-a74b-9e17fc9d1784:209142
11/09/2020 10:03:09 [15627] [WBXML] I </ComposeMail:ItemId>
becomes
[itemid] => 209142
and has lost the relevant containing folder identifier.

11/09/2020 10:03:09 [15627] [WBXML] I  <ComposeMail:SmartForward>
11/09/2020 10:03:09 [15627] [WBXML] I   <ComposeMail:ClientId>
11/09/2020 10:03:09 [15627] [WBXML] I    SendMail-519810345307892
11/09/2020 10:03:09 [15627] [WBXML] I   </ComposeMail:ClientId>
11/09/2020 10:03:09 [15627] [WBXML] I   <ComposeMail:SaveInSentItems>
11/09/2020 10:03:09 [15627] [WBXML] I    
11/09/2020 10:03:09 [15627] [WBXML] I   </ComposeMail:SaveInSentItems>
11/09/2020 10:03:09 [15627] [WBXML] I   <ComposeMail:Source>
11/09/2020 10:03:09 [15627] [WBXML] I    <ComposeMail:FolderId>
11/09/2020 10:03:09 [15627] [WBXML] I     FL0-2
11/09/2020 10:03:09 [15627] [WBXML] I    </ComposeMail:FolderId>
11/09/2020 10:03:09 [15627] [WBXML] I    <ComposeMail:ItemId>
11/09/2020 10:03:09 [15627] [WBXML] I     62635aaf-db95-42a1-a74b-9e17fc9d1784:209142
11/09/2020 10:03:09 [15627] [WBXML] I    </ComposeMail:ItemId>
11/09/2020 10:03:09 [15627] [WBXML] I   </ComposeMail:Source>
11/09/2020 10:03:09 [15627] [WBXML] I   <ComposeMail:MIME>

---- MIME deleted ----

11/09/2020 10:03:09 [15627] [WBXML] I   </ComposeMail:MIME>
11/09/2020 10:03:09 [15627] [WBXML] I  </ComposeMail:SmartForward>
11/09/2020 10:03:09 [15627] [DEBUG] DeviceManager->GetBackendIdForFolderId(): no backend-folderid available for 'FL0-2', returning as is.
11/09/2020 10:03:09 [15627] [DEBUG] Zimbra->SendMail(): START SendMail { $syncsm = (excluded); }
11/09/2020 10:03:09 [15627] [DEBUG] Zimbra->SendMail(): SyncSm [SyncSendMail Object
(
    [clientid] => SendMail-519810345307892
    [saveinsent] => 
    [replacemime] => 
    [accountid] => 
    [source] => SyncSendMailSource Object
        (
            [folderid] => FL0-2
            [itemid] => 209142
            [longid] => 
            [instanceid] => 
            [unsetVars:protected] => Array
                (
                )

            [supportsPrivateStripping:protected] => 
            [mapping:protected] => Array
                (
                    [ComposeMail:FolderId] => Array
                        (
                            [1] => folderid
                        )

                    [ComposeMail:ItemId] => Array
                        (
                            [1] => itemid
                        )

                    [ComposeMail:LongId] => Array
                        (
                            [1] => longid
                        )

                    [ComposeMail:InstanceId] => Array
                        (
                            [1] => instanceid
                        )

                )

            [flags] => 
            [content] => 
        )

    [mime] => SavedFromEmail:

Any ideas?

liverpoolfcfan

Replying to my own message. The issue is in SendMail.php at line 114 - adding debugs before and after shows

11/09/2020 10:45:25 [22328] [ERROR] SendMail(): Item id while replying or forwarding message:'62635aaf-db95-42a1-a74b-9e17fc9d1784:209142'
11/09/2020 10:45:25 [22328] [ERROR] SendMail(): Item id while replying or forwarding message:'209142'

            if (isset($sm->source->itemid)) {
ZLog::Write(LOGLEVEL_ERROR, sprintf("SendMail(): Item id while replying or forwarding message:'%s'", $sm->source->itemid));
                list(, $sk) = Utils::SplitMessageId($sm->source->itemid);
                $sm->source->itemid = $sk;
ZLog::Write(LOGLEVEL_ERROR, sprintf("SendMail(): Item id while replying or forwarding message:'%s'", $sm->source->itemid));
            }

What is the reason for this SplitMessageId?

Manfred

Hi @liverpoolfcfan,

@liverpoolfcfan said in SmartForward - itemid in ComposeMail object is changed:

What is the reason for this SplitMessageId?

Because it is folderid:messageid and folderid is in the FolderId tag (saved as $sm->source->folderid).

Manfred

liverpoolfcfan

@Manfred said in SmartForward - itemid in ComposeMail object is changed:

Because it is folderid:messageid and folderid is in the FolderId tag (saved as $sm->source->folderid).

@Manfred That might be true for Kopano and/or Exchange but is not necessarily true for other email servers. In zimbra for instance, you can accept a share of another user’s mail folder. You will get a folder alias to that folder that is local to your user. However, if you list the mail items in there, they are presented to you as user_guid:item_id. You cannot access individual emails as your_folder_alias:item_id - you must use user_guid:item_id

You can see this in the log extract above. My local alias for the shared folder is FL0-2 but the mail item is identified as 62635aaf-db95-42a1-a74b-9e17fc9d1784:209142 which contains the actual user_guid of the owner and the item number - 209142 in this case.

If the item were in a folder owned by the user themselves, the item_id would simply be returned as 209142

Manfred

Hi @liverpoolfcfan,

so the issue is that zimbra uses the same “:” separator for user_guid and item_id. Is it possible to use a different one in Zimbra? Or is it possible for the zimbra backend to match the folder alias to user_guid if necessary? That would solve your issue.

Moving the split into backends goes against the logic as the backend themselves don’t care and don’t need the short backend id. Then also all the current backends would need to do the splitting themselves. I’m aware of kopano and imap backends and could change them, but everyone else who’s not hosting their email backend at z-hub.io (e.g. egroupware) would need to adapt them.

I know that it would be a simple change but due to its high impact that would be something for a major version (3.0 or 2.7.0).

Z-Push 2.6.0 is almost out, so I wouldn’t like to do such a change so close to the release.

Manfred

liverpoolfcfan

@Manfred It is not possible to change the separator in zimbra. It is what it is.

I think I can identify the user_guid from the folder alias to workaround the issue. I will let you know.

liverpoolfcfan

@Manfred In the example I provided above you can see that there is no visible correlation between the FolderId and the ItemId.

Is it the case that for Kopano/IMAP that you would see the same FolderId prepended to the ItemID in the incoming XML for the SmartReply/SmartForward request?

If you refer to this as a long form ItemID and short form ItemID, is the API not capable of accepting both?

Manfred

Hi @liverpoolfcfan,

@liverpoolfcfan said in SmartForward - itemid in ComposeMail object is changed:

@Manfred In the example I provided above you can see that there is no visible correlation between the FolderId and the ItemId.

And there’s also no mapping in zimbra backend between the folderid and user_guid?

Is it the case that for Kopano/IMAP that you would see the same FolderId prepended to the ItemID in the incoming XML for the SmartReply/SmartForward request?

Yes, the folderid is prepended to the itemid.

If you refer to this as a long form ItemID and short form ItemID, is the API not capable of accepting both?

Of course it would be possible to leave the ComposeMail:ItemId as is in the $sm object and let the backends handle $sm->source->itemid as they like. However Kopano and IMAP backends (and maybe also some other custom backends) would then need to split $sm->source->itemid in folderid and itemid themselves which is also not ideal. As I said in my previous post it would be a change for a major version only if we decide to do that.

Manfred

liverpoolfcfan

@Manfred said in SmartForward - itemid in ComposeMail object is changed:

I can get the owner_guid by checking the link associated with the local folder alias. It provides owner_guid:realFolderId - so I can handle a workaround for now

My question was related to

However Kopano and IMAP backends (and maybe also some other custom backends) would then need to split $sm->source->itemid in folderid and itemid themselves which is also not ideal.

Do they “have to” split the folderId and itemId? Would the APIs for fetching/moving/etc not accept a long form folderId:itemId

Manfred

Hi @liverpoolfcfan,

@liverpoolfcfan said in SmartForward - itemid in ComposeMail object is changed:

@Manfred said in SmartForward - itemid in ComposeMail object is changed:
My question was related to

However Kopano and IMAP backends (and maybe also some other custom backends) would then need to split $sm->source->itemid in folderid and itemid themselves which is also not ideal.

Do they “have to” split the folderId and itemId? Would the APIs for fetching/moving/etc not accept a long form folderId:itemId

The Fetch command accepts the longid, but SendMail doesn’t. Maybe it’s different for zimbra, but Kopano and IMAP backends have to split the folderid from the itemid at some point.

Manfred

liverpoolfcfan

@Manfred OK. Thanks. I will implement a workaround for now.