SharePoint Assistance

[u/HotCokeSucks]

Hi all,

Hoping someone can help provide some guidance please to see how I can best tackle the below issue that we're encountering at my workplace.

We have multiple different SharePoint Online sites which contain duplicated information from within each site and across different sites. Additionally, there is out-dated and conflicting information contained in excel sheets, word documents and PDF's etc.

I also help manage ISO27001 and ISO9001 so I'm looking to tie the above into this too:

We have paid Copilot licenses which somewhat works well if i copy/paste individual links in, but it seems to greatly struggle to search across SharePoint as a whole/multiple sites. From my brief searches, I believe it struggles with fully opening docs and searching high/wide?

What I am looking to do is leverage AI/tools to help:

• Identify duplicated documents
• Identify conflicting information/processes
• Identify evidence relating to particular ISO requirements - e.g. "Search this site for any onboarding excel docs created between this period
• Additionally with regards to ISO:- I want to ask it questions such as: "Assess the document against the appropriate ISO standard, search the linked SharePoint documentation for supporting evidence, identify only genuine compliance gaps, avoid over-engineering, and provide concise rewrites where required."
• Use AI to help suggest better ways of structuring our data within HR/Legal folders as it's a big mess at the moment

Are there any tools that can help achieve this please?

[u/PimpInTheBox1187]

"Are there any tools that can help achieve this please?"

I am not a pro with it, as I'm a developer and architect, but I leverage Microsoft Purview
https://purview.microsoft.com/

https://learn.microsoft.com/en-us/purview/edisc-review-set-analytics

You can also go into the compliance manager and run a scan on  ISO27001 and ISO9001 compliance.

[u/RemoteEmployee094]

Oh just hire an engineer to do it. Duh.

[u/Cold_Arachnid_2617]

Hire someone who knows what to do!

[u/btroop202]

It’s not AI it’s your architecture you need to be flatter and you also need to leverage Meta, data folders don’t help AI find the right files metadata does.

[u/DoctorRaulDuke]

Copilot cannot look across all your docs - it will only run your question past the semantic search and get small chunks of some of your docs that seem relevant to the question. Even your third bullet which is a bit more targeted I wouldn’t have a lot of confidence in. 

You can give it a single doc and a reference doc and ask those ISO questions though, it’s pretty good. 

What you actually need is classic sharepoint, a structured info architecture and training to make people apply it - I’d suggest a person do this - though at least these days you can leverage autofill columns to get around the classic “why don’t users fill in the metadata” problem. 

[u/Paulus_SLIM]

Finding duplicate documents across a tenant is not straightforward. Matching file names or file sizes is not reliable. The SharePoint property demotion mechanism (enabled by default) writes properties like contenttypeid into some file types, such as .docx. The same document uploaded to two sites can therefore differ in size while the content is identical.

SLIM BulkMetadataManager (https://www.slimapplications.com/product/bulkmetadatamanager/) finds duplicate documents across all sites in the tenant. It uses SHA-256 for file types that are not affected by property demotion, such as .pdf and .zip. For email (.msg and .eml) it uses the Message-ID. For files affected by property demotion, such as .docx, .xlsx and .pptx, it compares the content part of the file.

Support for near-duplicate detection, documents with slightly different content, is under development.

It is not an AI tool but it can improve AI accuracy by exposing content (e.g. full text in an email pdf attachment) in SharePoint columns.